Deprecating kibana_user and kibana_dashboard_only_user roles (#50963)

This change adds a new `kibana_admin` role, and deprecates
the old `kibana_user` and`kibana_dashboard_only_user`roles.

The deprecation is implemented via a new reserved metadata
attribute, which can be consumed from the API and also triggers
deprecation logging when used (by a user authenticating to
Elasticsearch).

Some docs have been updated to avoid references to these
deprecated roles.

Backport of: #46456

Co-authored-by: Larry Gregory <lgregorydev@gmail.com>

client/rest-high-level/src/test/java/org/elasticsearch/client/documentation/SecurityDocumentationIT.java

@@ -693,8 +693,8 @@ public class SecurityDocumentationIT extends ESRestHighLevelClientTestCase {
 
             List<Role> roles = response.getRoles();
             assertNotNull(response);
-            // 28 system roles plus the three we created
+            // 29 system roles plus the three we created
-            assertThat(roles.size(), equalTo(28 + 3));
+            assertThat(roles.size(), equalTo(29 + 3));
         }
 
         {

docs/reference/monitoring/configuring-filebeat.asciidoc

@@ -117,7 +117,7 @@ If {security-features} are enabled, you must provide a valid user ID and
 password so that {filebeat} can connect to {kib}: 
 
 .. Create a user on the monitoring cluster that has the 
-<<built-in-roles,`kibana_user` built-in role>> or equivalent
+<<built-in-roles,`kibana_admin` built-in role>> or equivalent
 privileges.
 
 .. Add the `username` and `password` settings to the {es} output information in 

x-pack/docs/en/security/authentication/oidc-guide.asciidoc

@@ -418,14 +418,14 @@ through either the
 NOTE: You cannot use <<mapping-roles-file,role mapping files>>
 to grant roles to users authenticating via OpenID Connect.
 
-This is an example of a simple role mapping that grants the `kibana_user` role
+This is an example of a simple role mapping that grants the `example_role` role
 to any user who authenticates against the `oidc1` OpenID Connect realm:
 
 [source,console]
 --------------------------------------------------
-PUT /_security/role_mapping/oidc-kibana
+PUT /_security/role_mapping/oidc-example
 {
-  "roles": [ "kibana_user" ],
+  "roles": [ "example_role" ], <1>
   "enabled": true,
   "rules": {
     "field": { "realm.name": "oidc1" }
@@ -433,6 +433,10 @@ PUT /_security/role_mapping/oidc-kibana
 }
 --------------------------------------------------
 
+<1> The `example_role` role is *not* a builtin Elasticsearch role.
+This example assumes that you have created a custom role of your own, with
+appropriate access to your <<roles-indices-priv,indices>> and
+{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features].
 
 The user properties that are mapped via the realm configuration are used to process
 role mapping rules, and these rules determine which roles a user is granted.

x-pack/docs/en/security/authentication/saml-guide.asciidoc

@@ -631,14 +631,14 @@ through either the
 NOTE: You cannot use <<mapping-roles-file,role mapping files>>
 to grant roles to users authenticating via SAML.
 
-This is an example of a simple role mapping that grants the `kibana_user` role
+This is an example of a simple role mapping that grants the `example_role` role
 to any user who authenticates against the `saml1` realm:
 
 [source,console]
 --------------------------------------------------
-PUT /_security/role_mapping/saml-kibana
+PUT /_security/role_mapping/saml-example
 {
-  "roles": [ "kibana_user" ],
+  "roles": [ "example_role" ], <1>
   "enabled": true,
   "rules": {
     "field": { "realm.name": "saml1" }
@@ -646,6 +646,10 @@ PUT /_security/role_mapping/saml-kibana
 }
 --------------------------------------------------
 
+<1> The `example_role` role is *not* a builtin Elasticsearch role.
+This example assumes that you have created a custom role of your own, with
+appropriate access to your <<roles-indices-priv,indices>> and
+{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[Kibana features].
 
 The attributes that are mapped via the realm configuration are used to process
 role mapping rules, and these rules determine which roles a user is granted.

x-pack/docs/en/security/authorization/built-in-roles.asciidoc

@@ -72,10 +72,12 @@ NOTE: This role does *not* provide the ability to create indices; those privileg
 must be defined in a separate role.
 
 [[built-in-roles-kibana-dashboard]] `kibana_dashboard_only_user` ::
-Grants access to the {kib} Dashboard and read-only permissions to Kibana.
+(This role is deprecated, please use 
-This role does not have access to editing tools in {kib}. For more
+{kibana-ref}/kibana-privileges.html#kibana-feature-privileges[{kib} feature privileges]
-information, see
+instead).
-{kibana-ref}/xpack-dashboard-only-mode.html[{kib} Dashboard Only Mode].
+Grants read-only access to the {kib} Dashboard in every 
+{kibana-ref}/xpack-spaces.html[space in {kib}].
+This role does not have access to editing tools in {kib}. 
 
 [[built-in-roles-kibana-system]] `kibana_system` ::
 Grants access necessary for the {kib} system user to read from and write to the
@@ -87,9 +89,15 @@ see {kibana-ref}/using-kibana-with-security.html[Configuring Security in {kib}].
 NOTE: This role should not be assigned to users as the granted permissions may
 change between releases.
 
+[[built-in-roles-kibana-admin]] `kibana_admin`::
+Grants access to all features in {kib}. For more information on {kib} authorization,
+see {kibana-ref}/xpack-security-authorization.html[Kibana authorization].
+
 [[built-in-roles-kibana-user]] `kibana_user`::
-Grants access to all features in {kib}. For more information on Kibana authorization,
+(This role is deprecated, please use the
-see {kibana-ref}/xpack-security-authorization.html[Kibana Authorization].
+<<built-in-roles-kibana-admin,`kibana_admin`>> role instead.)
+Grants access to all features in {kib}. For more information on {kib} authorization,
+see {kibana-ref}/xpack-security-authorization.html[Kibana authorization].
 
 [[built-in-roles-logstash-admin]] `logstash_admin` ::
 Grants access to the `.logstash*` indices for managing configurations.
@@ -127,7 +135,8 @@ Grants the minimum privileges required for any user of {monitoring} other than t
 required to use {kib}. This role grants access to the monitoring indices and grants
 privileges necessary for reading basic cluster information. This role also includes
 all {kibana-ref}/kibana-privileges.html[Kibana privileges] for the {stack-monitor-features}.
-Monitoring users should also be assigned the `kibana_user` role.
+Monitoring users should also be assigned the `kibana_admin` role, or another role
+with {kibana-ref}/xpack-security-authorization.html[access to the {kib} instance].
 
 [[built-in-roles-remote-monitoring-agent]] `remote_monitoring_agent`::
 Grants the minimum privileges required to write data into the monitoring indices
@@ -140,9 +149,10 @@ Grants the minimum privileges required to collect monitoring data for the {stack
 [[built-in-roles-reporting-user]] `reporting_user`::
 Grants the specific privileges required for users of {reporting} other than those
 required to use {kib}. This role grants access to the reporting indices; each 
-user has access to only their own reports. Reporting users should also be 
+user has access to only their own reports.
-assigned the `kibana_user` role and a role that grants them access to the data 
+Reporting users should also be assigned additional roles that grant 
-that will be used to generate reports.
+{kibana-ref}/xpack-security-authorization.html[access to {kib}] as well as read
+access to the <<roles-indices-priv,indices>> that will be used to generate reports.
 
 [[built-in-roles-snapshot-user]] `snapshot_user`::
 Grants the necessary privileges to create snapshots of **all** the indices and

x-pack/docs/en/security/ccs-clients-integrations/cross-cluster-kibana.asciidoc

@@ -31,8 +31,9 @@ NOTE: If you configure the local cluster as another remote in {es}, the
 `logstash_reader` role on your local cluster also needs to grant the
 `read_cross_cluster` privilege.
 
-. Assign your {kib} users the `kibana_user` role and your `logstash_reader`
+. Assign your {kib} users a role that grants
-role.
+{kibana-ref}/xpack-security-authorization.html[access to {kib}]
+as well as your `logstash_reader` role.
 
 . On the remote cluster, create a `logstash_reader` role that grants the
 `read_cross_cluster` privilege and `read` and `view_index_metadata` privileges

x-pack/docs/en/security/get-started-security.asciidoc

@@ -167,15 +167,16 @@ Select a role to see more information about its privileges. For example, select
 the `kibana_system` role to see its list of cluster and index privileges. To
 learn more, see <<privileges-list-indices>>. 
 
-Let's assign the `kibana_user` role to your user. Go back to the 
+Let's assign the `kibana_admin` role to your user. Go back to the 
-*Management / Security / Users* page and select your user. Add the `kibana_user` 
+*Management / Security / Users* page and select your user. Add the `kibana_admin` 
 role and save the change. For example:
 
 [role="screenshot"]
 image::security/images/assign-role.jpg["Assigning a role to a user in Kibana"]
 
-This user now has access to all features in {kib}. For more information about granting
+This user now has administrative access to all features in {kib}.
-access to Kibana see {kibana-ref}/xpack-security-authorization.html[Kibana Authorization].
+For more information about granting access to Kibana see
+{kibana-ref}/xpack-security-authorization.html[Kibana authorization].
 
 If you completed all of the steps in 
 {stack-gs}/get-started-elastic-stack.html[Getting started with the {stack}], you should 

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStore.java

@@ -52,11 +52,9 @@ public class ReservedRolesStore implements BiConsumer<Set<String>, ActionListene
                 .put("superuser", SUPERUSER_ROLE_DESCRIPTOR)
                 .put("transport_client", new RoleDescriptor("transport_client", new String[] { "transport_client" }, null, null,
                         MetadataUtils.DEFAULT_RESERVED_METADATA))
-                .put("kibana_user", new RoleDescriptor("kibana_user", null, null, new RoleDescriptor.ApplicationResourcePrivileges[] {
+                .put("kibana_admin", kibanaAdminUser("kibana_admin", MetadataUtils.DEFAULT_RESERVED_METADATA))
-                        RoleDescriptor.ApplicationResourcePrivileges.builder()
+                .put("kibana_user", kibanaAdminUser("kibana_user",
-                            .application("kibana-.kibana").resources("*").privileges("all").build() },
+                    MetadataUtils.getDeprecatedReservedMetadata("Please use the [kibana_admin] role instead")))
-                        null, null,
-                        MetadataUtils.DEFAULT_RESERVED_METADATA, null))
                 .put("monitoring_user", new RoleDescriptor("monitoring_user",
                         new String[] { "cluster:monitor/main", "cluster:monitor/xpack/info", RemoteInfoAction.NAME },
                         new RoleDescriptor.IndicesPrivileges[] {
@@ -110,7 +108,7 @@ public class ReservedRolesStore implements BiConsumer<Set<String>, ActionListene
                             RoleDescriptor.ApplicationResourcePrivileges.builder()
                             .application("kibana-.kibana").resources("*").privileges("read").build() },
                         null, null,
-                        MetadataUtils.DEFAULT_RESERVED_METADATA,
+                        MetadataUtils.getDeprecatedReservedMetadata("Please use Kibana feature privileges instead"),
                         null))
                 .put(KibanaUser.ROLE_NAME, new RoleDescriptor(KibanaUser.ROLE_NAME,
                         new String[] {
@@ -266,6 +264,16 @@ public class ReservedRolesStore implements BiConsumer<Set<String>, ActionListene
                 .immutableMap();
     }
 
+    private static RoleDescriptor kibanaAdminUser(String name, Map<String, Object> metadata) {
+        return new RoleDescriptor(name, null, null,
+            new RoleDescriptor.ApplicationResourcePrivileges[] {
+                RoleDescriptor.ApplicationResourcePrivileges.builder()
+                    .application("kibana-.kibana")
+                    .resources("*").privileges("all")
+                    .build() },
+            null, null, metadata, null);
+    }
+
     public static boolean isReserved(String role) {
         return RESERVED_ROLES.containsKey(role) || UsernamesField.SYSTEM_ROLE.equals(role) || UsernamesField.XPACK_ROLE.equals(role);
     }

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/support/MetadataUtils.java

@@ -5,6 +5,8 @@
  */
 package org.elasticsearch.xpack.core.security.support;
 
+import org.elasticsearch.common.collect.MapBuilder;
+
 import java.util.Collections;
 import java.util.Map;
 
@@ -12,6 +14,8 @@ public class MetadataUtils {
 
     public static final String RESERVED_PREFIX = "_";
     public static final String RESERVED_METADATA_KEY = RESERVED_PREFIX + "reserved";
+    public static final String DEPRECATED_METADATA_KEY = RESERVED_PREFIX + "deprecated";
+    public static final String DEPRECATED_REASON_METADATA_KEY = RESERVED_PREFIX + "deprecated_reason";
     public static final Map<String, Object> DEFAULT_RESERVED_METADATA = Collections.singletonMap(RESERVED_METADATA_KEY, true);
 
     private MetadataUtils() {
@@ -25,4 +29,12 @@ public class MetadataUtils {
         }
         return false;
     }
+
+    public static Map<String, Object> getDeprecatedReservedMetadata(String reason) {
+        return MapBuilder.<String, Object>newMapBuilder()
+            .put(RESERVED_METADATA_KEY, true)
+            .put(DEPRECATED_METADATA_KEY, true)
+            .put(DEPRECATED_REASON_METADATA_KEY, reason)
+            .immutableMap();
+    }
 }

x-pack/plugin/core/src/test/java/org/elasticsearch/xpack/core/security/authz/store/ReservedRolesStoreTests.java

@@ -169,6 +169,7 @@ import java.util.SortedMap;
 
 import static org.hamcrest.Matchers.hasEntry;
 import static org.hamcrest.Matchers.is;
+import static org.hamcrest.Matchers.not;
 import static org.mockito.Mockito.mock;
 
 /**
@@ -184,6 +185,7 @@ public class ReservedRolesStoreTests extends ESTestCase {
         assertThat(ReservedRolesStore.isReserved("foobar"), is(false));
         assertThat(ReservedRolesStore.isReserved(SystemUser.ROLE_NAME), is(true));
         assertThat(ReservedRolesStore.isReserved("transport_client"), is(true));
+        assertThat(ReservedRolesStore.isReserved("kibana_admin"), is(true));
         assertThat(ReservedRolesStore.isReserved("kibana_user"), is(true));
         assertThat(ReservedRolesStore.isReserved("ingest_admin"), is(true));
         assertThat(ReservedRolesStore.isReserved("monitoring_user"), is(true));
@@ -409,6 +411,54 @@ public class ReservedRolesStoreTests extends ESTestCase {
         assertNoAccessAllowed(kibanaRole, RestrictedIndicesNames.ASYNC_SEARCH_PREFIX + randomAlphaOfLengthBetween(0, 2));
     }
 
+    public void testKibanaAdminRole() {
+        final TransportRequest request = mock(TransportRequest.class);
+        final Authentication authentication = mock(Authentication.class);
+
+        RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("kibana_admin");
+        assertNotNull(roleDescriptor);
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+        assertThat(roleDescriptor.getMetadata(), not(hasEntry("_deprecated", true)));
+
+        Role kibanaAdminRole = Role.builder(roleDescriptor, null).build();
+        assertThat(kibanaAdminRole.cluster().check(ClusterHealthAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterStateAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterStatsAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(PutIndexTemplateAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterRerouteAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(ClusterUpdateSettingsAction.NAME, request, authentication),
+                is(false));
+        assertThat(kibanaAdminRole.cluster().check(MonitoringBulkAction.NAME, request, authentication), is(false));
+        assertThat(kibanaAdminRole.cluster().check(DelegatePkiAuthenticationAction.NAME, request, authentication),
+                is(false));
+
+        assertThat(kibanaAdminRole.runAs().check(randomAlphaOfLengthBetween(1, 12)), is(false));
+
+        assertThat(kibanaAdminRole.indices().allowedIndicesMatcher(IndexAction.NAME).test("foo"), is(false));
+        assertThat(kibanaAdminRole.indices().allowedIndicesMatcher(IndexAction.NAME).test(".reporting"), is(false));
+        assertThat(
+                kibanaAdminRole.indices().allowedIndicesMatcher("indices:foo").test(randomAlphaOfLengthBetween(8, 24)),
+                is(false));
+
+        final String randomApplication = "kibana-" + randomAlphaOfLengthBetween(8, 24);
+        assertThat(kibanaAdminRole.application().grants(new ApplicationPrivilege(randomApplication, "app-random", "all"),
+                "*"), is(false));
+
+        final String application = "kibana-.kibana";
+        assertThat(kibanaAdminRole.application().grants(new ApplicationPrivilege(application, "app-foo", "foo"), "*"),
+                is(false));
+        assertThat(kibanaAdminRole.application().grants(new ApplicationPrivilege(application, "app-all", "all"), "*"),
+                is(true));
+
+        final String applicationWithRandomIndex = "kibana-.kibana_" + randomAlphaOfLengthBetween(8, 24);
+        assertThat(
+                kibanaAdminRole.application()
+                        .grants(new ApplicationPrivilege(applicationWithRandomIndex, "app-random-index", "all"), "*"),
+                is(false));
+
+        assertNoAccessAllowed(kibanaAdminRole, RestrictedIndicesNames.RESTRICTED_NAMES);
+    }
+
     public void testKibanaUserRole() {
         final TransportRequest request = mock(TransportRequest.class);
         final Authentication authentication = mock(Authentication.class);
@@ -416,6 +466,7 @@ public class ReservedRolesStoreTests extends ESTestCase {
         RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("kibana_user");
         assertNotNull(roleDescriptor);
         assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_deprecated", true));
 
         Role kibanaUserRole = Role.builder(roleDescriptor, null).build();
         assertThat(kibanaUserRole.cluster().check(ClusterHealthAction.NAME, request, authentication), is(false));
@@ -745,6 +796,7 @@ public class ReservedRolesStoreTests extends ESTestCase {
         RoleDescriptor roleDescriptor = new ReservedRolesStore().roleDescriptor("kibana_dashboard_only_user");
         assertNotNull(roleDescriptor);
         assertThat(roleDescriptor.getMetadata(), hasEntry("_reserved", true));
+        assertThat(roleDescriptor.getMetadata(), hasEntry("_deprecated", true));
 
         Role dashboardsOnlyUserRole = Role.builder(roleDescriptor, null).build();
         assertThat(dashboardsOnlyUserRole.cluster().check(ClusterHealthAction.NAME, request, authentication), is(false));

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/authz/store/CompositeRolesStore.java

@@ -17,6 +17,7 @@ import org.elasticsearch.common.bytes.BytesReference;
 import org.elasticsearch.common.cache.Cache;
 import org.elasticsearch.common.cache.CacheBuilder;
 import org.elasticsearch.common.collect.Tuple;
+import org.elasticsearch.common.logging.DeprecationLogger;
 import org.elasticsearch.common.settings.Setting;
 import org.elasticsearch.common.settings.Setting.Property;
 import org.elasticsearch.common.settings.Settings;
@@ -41,6 +42,7 @@ import org.elasticsearch.xpack.core.security.authz.privilege.Privilege;
 import org.elasticsearch.xpack.core.security.authz.store.ReservedRolesStore;
 import org.elasticsearch.xpack.core.security.authz.store.RoleRetrievalResult;
 import org.elasticsearch.xpack.core.security.support.CacheIteratorHelper;
+import org.elasticsearch.xpack.core.security.support.MetadataUtils;
 import org.elasticsearch.xpack.core.security.user.AnonymousUser;
 import org.elasticsearch.xpack.core.security.user.SystemUser;
 import org.elasticsearch.xpack.core.security.user.User;
@@ -83,6 +85,7 @@ public class CompositeRolesStore {
         Setting.intSetting("xpack.security.authz.store.roles.negative_lookup_cache.max_size", 10000, Property.NodeScope);
     private static final Logger logger = LogManager.getLogger(CompositeRolesStore.class);
 
+    private final DeprecationLogger deprecationLogger = new DeprecationLogger(logger);
 
     private final FileRolesStore fileRolesStore;
     private final NativeRolesStore nativeRolesStore;
@@ -154,6 +157,7 @@ public class CompositeRolesStore {
             final long invalidationCounter = numInvalidation.get();
             roleDescriptors(roleNames, ActionListener.wrap(
                     rolesRetrievalResult -> {
+                        logDeprecatedRoles(rolesRetrievalResult.roleDescriptors);
                         final boolean missingRoles = rolesRetrievalResult.getMissingRoles().isEmpty() == false;
                         if (missingRoles) {
                             logger.debug(() -> new ParameterizedMessage("Could not find roles with names {}",
@@ -179,6 +183,17 @@ public class CompositeRolesStore {
         }
     }
 
+    void logDeprecatedRoles(Set<RoleDescriptor> roleDescriptors) {
+        roleDescriptors.stream()
+            .filter(rd -> Boolean.TRUE.equals(rd.getMetadata().get(MetadataUtils.DEPRECATED_METADATA_KEY)))
+            .forEach(rd -> {
+                String reason = Objects.toString(
+                    rd.getMetadata().get(MetadataUtils.DEPRECATED_REASON_METADATA_KEY), "Please check the documentation");
+                deprecationLogger.deprecatedAndMaybeLog("deprecated_role-" + rd.getName(), "The role [" + rd.getName() +
+                    "] is deprecated and will be removed in a future version of Elasticsearch. " + reason);
+            });
+    }
+
     public void getRoles(User user, Authentication authentication, ActionListener<Role> roleActionListener) {
         // we need to special case the internal users in this method, if we apply the anonymous roles to every user including these system
         // user accounts then we run into the chance of a deadlock because then we need to get a role that we may be trying to get as the

x-pack/plugin/security/src/test/java/org/elasticsearch/xpack/security/authz/store/CompositeRolesStoreTests.java

@@ -15,6 +15,7 @@ import org.elasticsearch.action.support.PlainActionFuture;
 import org.elasticsearch.cluster.health.ClusterHealthStatus;
 import org.elasticsearch.cluster.metadata.IndexMetaData;
 import org.elasticsearch.cluster.metadata.MetaData;
+import org.elasticsearch.common.Nullable;
 import org.elasticsearch.common.bytes.BytesReference;
 import org.elasticsearch.common.io.stream.StreamOutput;
 import org.elasticsearch.common.settings.Settings;
@@ -53,6 +54,7 @@ import org.elasticsearch.xpack.core.security.authz.privilege.IndexPrivilege;
 import org.elasticsearch.xpack.core.security.authz.store.ReservedRolesStore;
 import org.elasticsearch.xpack.core.security.authz.store.RoleRetrievalResult;
 import org.elasticsearch.xpack.core.security.index.RestrictedIndicesNames;
+import org.elasticsearch.xpack.core.security.support.MetadataUtils;
 import org.elasticsearch.xpack.core.security.user.AnonymousUser;
 import org.elasticsearch.xpack.core.security.user.SystemUser;
 import org.elasticsearch.xpack.core.security.user.User;
@@ -64,11 +66,14 @@ import org.elasticsearch.xpack.security.support.SecurityIndexManager;
 
 import java.io.IOException;
 import java.time.Instant;
+import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.HashSet;
+import java.util.LinkedHashSet;
+import java.util.List;
 import java.util.Map;
 import java.util.Set;
 import java.util.concurrent.ExecutionException;
@@ -85,6 +90,7 @@ import static org.hamcrest.Matchers.anyOf;
 import static org.hamcrest.Matchers.containsInAnyOrder;
 import static org.hamcrest.Matchers.equalTo;
 import static org.hamcrest.Matchers.hasItem;
+import static org.hamcrest.Matchers.hasSize;
 import static org.hamcrest.Matchers.is;
 import static org.hamcrest.Matchers.nullValue;
 import static org.mockito.Matchers.any;
@@ -144,21 +150,14 @@ public class CompositeRolesStoreTests extends ESTestCase {
         }, null);
         FileRolesStore fileRolesStore = mock(FileRolesStore.class);
         doCallRealMethod().when(fileRolesStore).accept(any(Set.class), any(ActionListener.class));
-        ReservedRolesStore reservedRolesStore = mock(ReservedRolesStore.class);
-        doCallRealMethod().when(reservedRolesStore).accept(any(Set.class), any(ActionListener.class));
-        NativeRolesStore nativeRolesStore = mock(NativeRolesStore.class);
-        doCallRealMethod().when(nativeRolesStore).accept(any(Set.class), any(ActionListener.class));
 
         when(fileRolesStore.roleDescriptors(Collections.singleton("fls"))).thenReturn(Collections.singleton(flsRole));
         when(fileRolesStore.roleDescriptors(Collections.singleton("dls"))).thenReturn(Collections.singleton(dlsRole));
         when(fileRolesStore.roleDescriptors(Collections.singleton("fls_dls"))).thenReturn(Collections.singleton(flsDlsRole));
         when(fileRolesStore.roleDescriptors(Collections.singleton("no_fls_dls"))).thenReturn(Collections.singleton(noFlsDlsRole));
         final AtomicReference<Collection<RoleDescriptor>> effectiveRoleDescriptors = new AtomicReference<Collection<RoleDescriptor>>();
-        final DocumentSubsetBitsetCache documentSubsetBitsetCache = buildBitsetCache();
+        CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(Settings.EMPTY, fileRolesStore, null,
-        CompositeRolesStore compositeRolesStore = new CompositeRolesStore(Settings.EMPTY, fileRolesStore, nativeRolesStore,
+            null, null, licenseState, null, null, rds -> effectiveRoleDescriptors.set(rds));
-                reservedRolesStore, mock(NativePrivilegeStore.class), Collections.emptyList(),
-                new ThreadContext(Settings.EMPTY), licenseState, cache, mock(ApiKeyService.class), documentSubsetBitsetCache,
-                rds -> effectiveRoleDescriptors.set(rds));
 
         PlainActionFuture<Role> roleFuture = new PlainActionFuture<>();
         compositeRolesStore.roles(Collections.singleton("fls"), roleFuture);
@@ -221,20 +220,13 @@ public class CompositeRolesStoreTests extends ESTestCase {
         }, null);
         FileRolesStore fileRolesStore = mock(FileRolesStore.class);
         doCallRealMethod().when(fileRolesStore).accept(any(Set.class), any(ActionListener.class));
-        ReservedRolesStore reservedRolesStore = mock(ReservedRolesStore.class);
-        doCallRealMethod().when(reservedRolesStore).accept(any(Set.class), any(ActionListener.class));
-        NativeRolesStore nativeRolesStore = mock(NativeRolesStore.class);
-        doCallRealMethod().when(nativeRolesStore).accept(any(Set.class), any(ActionListener.class));
         when(fileRolesStore.roleDescriptors(Collections.singleton("fls"))).thenReturn(Collections.singleton(flsRole));
         when(fileRolesStore.roleDescriptors(Collections.singleton("dls"))).thenReturn(Collections.singleton(dlsRole));
         when(fileRolesStore.roleDescriptors(Collections.singleton("fls_dls"))).thenReturn(Collections.singleton(flsDlsRole));
         when(fileRolesStore.roleDescriptors(Collections.singleton("no_fls_dls"))).thenReturn(Collections.singleton(noFlsDlsRole));
         final AtomicReference<Collection<RoleDescriptor>> effectiveRoleDescriptors = new AtomicReference<Collection<RoleDescriptor>>();
-        final DocumentSubsetBitsetCache documentSubsetBitsetCache = buildBitsetCache();
+        CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(Settings.EMPTY, fileRolesStore, null,
-        CompositeRolesStore compositeRolesStore = new CompositeRolesStore(Settings.EMPTY, fileRolesStore, nativeRolesStore,
+            null, null, licenseState, null, null, rds -> effectiveRoleDescriptors.set(rds));
-                reservedRolesStore, mock(NativePrivilegeStore.class), Collections.emptyList(),
-                new ThreadContext(Settings.EMPTY), licenseState, cache, mock(ApiKeyService.class), documentSubsetBitsetCache,
-                rds -> effectiveRoleDescriptors.set(rds));
 
         PlainActionFuture<Role> roleFuture = new PlainActionFuture<>();
         compositeRolesStore.roles(Collections.singleton("fls"), roleFuture);
@@ -267,6 +259,7 @@ public class CompositeRolesStoreTests extends ESTestCase {
         final NativeRolesStore nativeRolesStore = mock(NativeRolesStore.class);
         doCallRealMethod().when(nativeRolesStore).accept(any(Set.class), any(ActionListener.class));
         when(fileRolesStore.roleDescriptors(anySetOf(String.class))).thenReturn(Collections.emptySet());
+
         doAnswer((invocationOnMock) -> {
             ActionListener<RoleRetrievalResult> callback = (ActionListener<RoleRetrievalResult>) invocationOnMock.getArguments()[1];
             callback.onResponse(RoleRetrievalResult.success(Collections.emptySet()));
@@ -282,12 +275,9 @@ public class CompositeRolesStoreTests extends ESTestCase {
         }).when(nativePrivilegeStore).getPrivileges(isA(Set.class), isA(Set.class), any(ActionListener.class));
 
         final AtomicReference<Collection<RoleDescriptor>> effectiveRoleDescriptors = new AtomicReference<Collection<RoleDescriptor>>();
-        final DocumentSubsetBitsetCache documentSubsetBitsetCache = buildBitsetCache();
+        final CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(SECURITY_ENABLED_SETTINGS,
-        final CompositeRolesStore compositeRolesStore =
+            fileRolesStore, nativeRolesStore, reservedRolesStore, nativePrivilegeStore, null, null, null,
-                new CompositeRolesStore(SECURITY_ENABLED_SETTINGS, fileRolesStore, nativeRolesStore, reservedRolesStore,
+            rds -> effectiveRoleDescriptors.set(rds));
-                        nativePrivilegeStore, Collections.emptyList(), new ThreadContext(SECURITY_ENABLED_SETTINGS),
-                        new XPackLicenseState(SECURITY_ENABLED_SETTINGS), cache, mock(ApiKeyService.class),
-                        documentSubsetBitsetCache, rds -> effectiveRoleDescriptors.set(rds));
         verify(fileRolesStore).addListener(any(Consumer.class)); // adds a listener in ctor
 
         final String roleName = randomAlphaOfLengthBetween(1, 10);
@@ -323,7 +313,7 @@ public class CompositeRolesStoreTests extends ESTestCase {
         if (getSuperuserRole && numberOfTimesToCall > 0) {
             // the superuser role was requested so we get the role descriptors again
             verify(reservedRolesStore, times(2)).accept(anySetOf(String.class), any(ActionListener.class));
-            verify(nativePrivilegeStore).getPrivileges(isA(Set.class),isA(Set.class), any(ActionListener.class));
+            verify(nativePrivilegeStore).getPrivileges(isA(Set.class), isA(Set.class), any(ActionListener.class));
         }
         verifyNoMoreInteractions(fileRolesStore, reservedRolesStore, nativeRolesStore, nativePrivilegeStore);
     }
@@ -423,9 +413,6 @@ public class CompositeRolesStoreTests extends ESTestCase {
         verifyNoMoreInteractions(fileRolesStore, reservedRolesStore, nativeRolesStore);
     }
 
-    private DocumentSubsetBitsetCache buildBitsetCache() {
-        return new DocumentSubsetBitsetCache(Settings.EMPTY, mock(ThreadPool.class));
-    }
 
     public void testCustomRolesProviders() {
         final FileRolesStore fileRolesStore = mock(FileRolesStore.class);
@@ -900,12 +887,9 @@ public class CompositeRolesStoreTests extends ESTestCase {
         }).when(nativeRolesStore).getRoleDescriptors(isA(Set.class), any(ActionListener.class));
         final ReservedRolesStore reservedRolesStore = spy(new ReservedRolesStore());
 
-        final DocumentSubsetBitsetCache documentSubsetBitsetCache = buildBitsetCache();
+        final CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(SECURITY_ENABLED_SETTINGS, fileRolesStore,
-        final CompositeRolesStore compositeRolesStore =
+            nativeRolesStore, reservedRolesStore, mock(NativePrivilegeStore.class), null, mock(ApiKeyService.class),
-            new CompositeRolesStore(SECURITY_ENABLED_SETTINGS, fileRolesStore, nativeRolesStore, reservedRolesStore,
+            null, null);
-                mock(NativePrivilegeStore.class), Collections.emptyList(), new ThreadContext(SECURITY_ENABLED_SETTINGS),
-                new XPackLicenseState(SECURITY_ENABLED_SETTINGS), cache, mock(ApiKeyService.class), documentSubsetBitsetCache,
-                rds -> {});
         verify(fileRolesStore).addListener(any(Consumer.class)); // adds a listener in ctor
 
         PlainActionFuture<Role> rolesFuture = new PlainActionFuture<>();
@@ -941,11 +925,8 @@ public class CompositeRolesStoreTests extends ESTestCase {
         }).when(nativeRolesStore).getRoleDescriptors(isA(Set.class), any(ActionListener.class));
         final ReservedRolesStore reservedRolesStore = spy(new ReservedRolesStore());
 
-        final DocumentSubsetBitsetCache documentSubsetBitsetCache = buildBitsetCache();
+        final CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(settings, fileRolesStore, nativeRolesStore,
-        final CompositeRolesStore compositeRolesStore =
+            reservedRolesStore, mock(NativePrivilegeStore.class), null, mock(ApiKeyService.class), null, null);
-            new CompositeRolesStore(settings, fileRolesStore, nativeRolesStore, reservedRolesStore,
-                mock(NativePrivilegeStore.class), Collections.emptyList(), new ThreadContext(settings),
-                new XPackLicenseState(settings), cache, mock(ApiKeyService.class), documentSubsetBitsetCache, rds -> {});
         verify(fileRolesStore).addListener(any(Consumer.class)); // adds a listener in ctor
 
         PlainActionFuture<Role> rolesFuture = new PlainActionFuture<>();
@@ -1128,11 +1109,9 @@ public class CompositeRolesStoreTests extends ESTestCase {
 
         final DocumentSubsetBitsetCache documentSubsetBitsetCache = buildBitsetCache();
 
-        final CompositeRolesStore compositeRolesStore =
+        final CompositeRolesStore compositeRolesStore = buildCompositeRolesStore(
-            new CompositeRolesStore(SECURITY_ENABLED_SETTINGS, fileRolesStore, nativeRolesStore, reservedRolesStore,
+            SECURITY_ENABLED_SETTINGS, fileRolesStore, nativeRolesStore, reservedRolesStore, null, null, mock(ApiKeyService.class),
-                mock(NativePrivilegeStore.class), Collections.emptyList(), new ThreadContext(SECURITY_ENABLED_SETTINGS),
+            documentSubsetBitsetCache, null);
-                new XPackLicenseState(SECURITY_ENABLED_SETTINGS), cache, mock(ApiKeyService.class), documentSubsetBitsetCache, rds -> {
-            });
 
         PlainActionFuture<Map<String, Object>> usageStatsListener = new PlainActionFuture<>();
         compositeRolesStore.usageStats(usageStatsListener);
@@ -1142,6 +1121,111 @@ public class CompositeRolesStoreTests extends ESTestCase {
         assertThat(usageStats.get("dls"), is(Collections.singletonMap("bit_set_cache", documentSubsetBitsetCache.usageStats())));
     }
 
+    public void testLoggingOfDeprecatedRoles() {
+        List<RoleDescriptor> descriptors = new ArrayList<>();
+        Function<Map<String, Object>, RoleDescriptor> newRole = metadata -> new RoleDescriptor(
+            randomAlphaOfLengthBetween(4, 9), generateRandomStringArray(5, 5, false, true),
+            null, null, null, null, metadata, null);
+
+        RoleDescriptor deprecated1 = newRole.apply(MetadataUtils.getDeprecatedReservedMetadata("some reason"));
+        RoleDescriptor deprecated2 = newRole.apply(MetadataUtils.getDeprecatedReservedMetadata("a different reason"));
+
+        // Can't use getDeprecatedReservedMetadata because `Map.of` doesn't accept null values,
+        // so we clone metadata with a real value and then remove that key
+        final Map<String, Object> nullReasonMetadata = new HashMap<>(deprecated2.getMetadata());
+        nullReasonMetadata.remove(MetadataUtils.DEPRECATED_REASON_METADATA_KEY);
+        assertThat(nullReasonMetadata.keySet(), hasSize(deprecated2.getMetadata().size() -1));
+        RoleDescriptor deprecated3 = newRole.apply(nullReasonMetadata);
+
+        descriptors.add(deprecated1);
+        descriptors.add(deprecated2);
+        descriptors.add(deprecated3);
+
+        for (int i = randomIntBetween(2, 10); i > 0; i--) {
+            // the non-deprecated metadata is randomly one of:
+            // {}, {_deprecated:null}, {_deprecated:false},
+            // {_reserved:true}, {_reserved:true,_deprecated:null}, {_reserved:true,_deprecated:false}
+            Map<String, Object> metadata = randomBoolean() ? Collections.emptyMap() : MetadataUtils.DEFAULT_RESERVED_METADATA;
+            if (randomBoolean()) {
+                metadata = new HashMap<>(metadata);
+                metadata.put(MetadataUtils.DEPRECATED_METADATA_KEY, randomBoolean() ? null : false);
+            }
+            descriptors.add(newRole.apply(metadata));
+        }
+        Collections.shuffle(descriptors, random());
+
+        final CompositeRolesStore compositeRolesStore =
+            buildCompositeRolesStore(SECURITY_ENABLED_SETTINGS, null, null, null, null, null, null, null, null);
+
+        // Use a LHS so that the random-shufle-order of the list is preserved
+        compositeRolesStore.logDeprecatedRoles(new LinkedHashSet<>(descriptors));
+
+        assertWarnings(
+            "The role [" + deprecated1.getName() + "] is deprecated and will be removed in a future version of Elasticsearch." +
+                " some reason",
+            "The role [" + deprecated2.getName() + "] is deprecated and will be removed in a future version of Elasticsearch." +
+                " a different reason",
+            "The role [" + deprecated3.getName() + "] is deprecated and will be removed in a future version of Elasticsearch." +
+                " Please check the documentation"
+        );
+    }
+
+    private CompositeRolesStore buildCompositeRolesStore(Settings settings,
+                                                         @Nullable FileRolesStore fileRolesStore,
+                                                         @Nullable NativeRolesStore nativeRolesStore,
+                                                         @Nullable ReservedRolesStore reservedRolesStore,
+                                                         @Nullable NativePrivilegeStore privilegeStore,
+                                                         @Nullable XPackLicenseState licenseState,
+                                                         @Nullable ApiKeyService apiKeyService,
+                                                         @Nullable DocumentSubsetBitsetCache documentSubsetBitsetCache,
+                                                         @Nullable Consumer<Collection<RoleDescriptor>> roleConsumer) {
+        if (fileRolesStore == null) {
+            fileRolesStore = mock(FileRolesStore.class);
+            doCallRealMethod().when(fileRolesStore).accept(any(Set.class), any(ActionListener.class));
+            when(fileRolesStore.roleDescriptors(anySetOf(String.class))).thenReturn(Collections.emptySet());
+        }
+        if (nativeRolesStore == null) {
+            nativeRolesStore = mock(NativeRolesStore.class);
+            doCallRealMethod().when(nativeRolesStore).accept(any(Set.class), any(ActionListener.class));
+            doAnswer((invocationOnMock) -> {
+                ActionListener<RoleRetrievalResult> callback = (ActionListener<RoleRetrievalResult>) invocationOnMock.getArguments()[1];
+                callback.onResponse(RoleRetrievalResult.failure(new RuntimeException("intentionally failed!")));
+                return null;
+            }).when(nativeRolesStore).getRoleDescriptors(isA(Set.class), any(ActionListener.class));
+        }
+        if (reservedRolesStore == null) {
+            reservedRolesStore = mock(ReservedRolesStore.class);
+            doCallRealMethod().when(reservedRolesStore).accept(any(Set.class), any(ActionListener.class));
+        }
+        if (privilegeStore == null) {
+            privilegeStore = mock(NativePrivilegeStore.class);
+            doAnswer((invocationOnMock) -> {
+                ActionListener<Collection<ApplicationPrivilegeDescriptor>> callback = null;
+                callback = (ActionListener<Collection<ApplicationPrivilegeDescriptor>>) invocationOnMock.getArguments()[2];
+                callback.onResponse(Collections.emptyList());
+                return null;
+            }).when(privilegeStore).getPrivileges(isA(Set.class), isA(Set.class), any(ActionListener.class));
+        }
+        if (licenseState == null) {
+            licenseState = new XPackLicenseState(settings);
+        }
+        if (apiKeyService == null) {
+            apiKeyService = mock(ApiKeyService.class);
+        }
+        if (documentSubsetBitsetCache == null) {
+            documentSubsetBitsetCache = buildBitsetCache();
+        }
+        if (roleConsumer == null) {
+            roleConsumer = rds -> { };
+        }
+        return new CompositeRolesStore(settings, fileRolesStore, nativeRolesStore, reservedRolesStore, privilegeStore,
+            Collections.emptyList(), new ThreadContext(settings), licenseState, cache, apiKeyService, documentSubsetBitsetCache,
+            roleConsumer);
+    }
+
+    private DocumentSubsetBitsetCache buildBitsetCache() {
+        return new DocumentSubsetBitsetCache(Settings.EMPTY, mock(ThreadPool.class));
+    }
     private static class InMemoryRolesProvider implements BiConsumer<Set<String>, ActionListener<RoleRetrievalResult>> {
         private final Function<Set<String>, RoleRetrievalResult> roleDescriptorsFunc;
 

x-pack/qa/oidc-op-tests/src/test/java/org/elasticsearch/xpack/security/authc/oidc/OpenIdConnectAuthIT.java

@@ -276,7 +276,7 @@ public class OpenIdConnectAuthIT extends ESRestTestCase {
         final Map<String, Object> map = callAuthenticateApiUsingAccessToken(accessToken);
         logger.info("Authentication with token Response: " + map);
         assertThat(map.get("username"), equalTo("alice"));
-        assertThat((List<?>) map.get("roles"), containsInAnyOrder("kibana_user", "auditor"));
+        assertThat((List<?>) map.get("roles"), containsInAnyOrder("kibana_admin", "auditor"));
 
         assertThat(map.get("metadata"), instanceOf(Map.class));
         final Map<?, ?> metadata = (Map<?, ?>) map.get("metadata");
@@ -374,7 +374,7 @@ public class OpenIdConnectAuthIT extends ESRestTestCase {
 
     private void setRoleMappings() throws IOException {
         Request createRoleMappingRequest = new Request("PUT", "/_security/role_mapping/oidc_kibana");
-        createRoleMappingRequest.setJsonEntity("{ \"roles\" : [\"kibana_user\"]," +
+        createRoleMappingRequest.setJsonEntity("{ \"roles\" : [\"kibana_admin\"]," +
             "\"enabled\": true," +
             "\"rules\": {" +
             "\"field\": { \"realm.name\": \"" + REALM_NAME + "\"}" +