From 8e5503baa8556ea2f98dba9201c790fd872f7db9 Mon Sep 17 00:00:00 2001 From: Misty Stanley-Jones Date: Tue, 4 Nov 2014 15:39:50 +1000 Subject: [PATCH] HBASE-12421 Clarify ACL concepts and best practices Signed-off-by: stack --- src/main/docbkx/security.xml | 773 ++++++++++++++++++----------------- 1 file changed, 387 insertions(+), 386 deletions(-) diff --git a/src/main/docbkx/security.xml b/src/main/docbkx/security.xml index 03fab9e33d5..d649f95188c 100644 --- a/src/main/docbkx/security.xml +++ b/src/main/docbkx/security.xml @@ -573,233 +573,232 @@ grant 'rest_server', 'RWCA' HBase has a simpler security model than relational databases, especially in terms of client operations. No distinction is made between an insert (new record) and update (of - existing record), for example, as both collapse down into a Put. Accordingly, the - important operations condense to four permissions: READ, WRITE, CREATE, and ADMIN. - - Operation To Permission Mapping - - - - - - Permission - Operation - - - - - - Read - Get - - - - Exists - - - - Scan - - - - Write - Put - - - - Delete - - - - IncrementColumnValue - - - - CheckAndDelete/Put - - - - Create - Create - - - - Alter - - - - Drop - - - - Bulk Load - - - - Admin - Enable/Disable - - - - Snapshot/Restore/Clone - - - - Split - - - - Flush - - - - Compact - - - - Major Compact - - - - Roll HLog - - - - Grant - - - - Revoke - - - - Shutdown - - - Execute - Execute coprocessor endpoints - - - -
- Permissions can be granted in any of the following scopes, though CREATE and ADMIN - permissions are effective only at table, namespace, and global scopes. - - - Namespace + existing record), for example, as both collapse down into a Put. +
+ Understanding Access Levels + HBase access levels are granted independently of each other and allow for different + types of operations at a given scope. + + Read (R) - can read data at the given scope + + + Write (W) - can write data at the given scope + + + Execute (X) - can execute coprocessor endpoints at the given + scope + + + Create (C) - can create tables or drop tables (even those + they did not create) at the given scope + + + Admin (A) - can perform cluster operations such as balancing + the cluster or assigning regions at the given scope + + + The possible scopes are: + + + Superuser - superusers can perform any operation available in + HBase, to any resource. The user who runs HBase on your cluster is a superuser, as + are any principals assigned to the configuration property + hbase.superuser in hbase-site.xml on the + HMaster. + + + Global - permissions granted at global + scope allow the admin to operate on all tables of the cluster. + + + Namespace - permissions granted at + namespace scope apply to all tables within a given + namespace. + + + Table - permissions granted at table + scope apply to data or metadata within a given table. + + + ColumnFamily - permissions granted at + ColumnFamily scope apply to cells within that + ColumnFamily. + + + Cell - permissions granted at cell scope + apply to that exact cell coordinate (key, value, timestamp). This allows for policy + evolution along with data. + To change an ACL on a specific cell, write an updated cell with new ACL to the + precise coordinates of the original. + If you have a multi-versioned schema and want to update ACLs on all visible + versions, you need to write new cells for all visible versions. The application + has complete control over policy evolution. + The exception to the above rule is append and + increment processing. Appends and increments can carry an ACL in the + operation. If one is included in the operation, then it will be applied to the + result of the append or increment. Otherwise, the ACL of + the existing cell you are appending to or incrementing is preserved. + + + The combination of access levels and scopes creates a matrix of possible access + levels that can be granted to a user. In a production environment, it is useful to think + of access levels in terms of what is needed to do a specific job. The following list + describes appropriate access levels for some common types of HBase users. It is + important not to grant more access than is required for a given user to perform their + required tasks. + + + Superusers - In a production system, only the HBase user should have superuser + access. In a development environment, an administrator may need superuser access in + order to quickly control and manage the cluster. However, this type of administrator + should usually be a Global Admin rather than a superuser. + + + Global Admins - A global admin can perform tasks and access every table in + HBase. In a typical production environment, an admin should not have Read or Write + permissions to data within tables. - Read: User can read any table in the namespace. + A global admin with Admin permissions can perform cluster-wide operations on + the cluster, such as balancing, assigning or unassigning regions, or calling an + explicit major compaction. This is an operations role. - Write: User can write to any table in the namespace. - - - Create: User can create tables in the namespace. - - - Admin: User can alter table attributes; add, alter, or drop column families; - and enable, disable, or drop the table. User can also trigger region - (re)assignments or relocation. + A global admin with Create permissions can create or drop any table within + HBase. This is more of a DBA-type role. + In a production environment, it is likely that different users will have only + one of Admin and Create permissions. + + In the current implementation, a Global Admin with Admin + permission can grant himself Read and Write permissions + on a table and gain access to that table's data. For this reason, only grant + Global Admin permissions to trusted user who actually need + them. + Also be aware that a Global Admin with Create + permission can perform a Put operation on the ACL table, simulating a + grant or revoke and circumventing the authorization + check for Global Admin permissions. + Due to these issues, be cautious with granting Global Admin + privileges. + - - - Table - - - Read: User can read from any column family in table - - - Write: User can write to any column family in table - - - Create: User can alter table attributes; add, alter, or drop column - families; and drop the table. - - - Admin: User can alter table attributes; add, alter, or drop column families; - and enable, disable, or drop the table. User can also trigger region - (re)assignments or relocation. - - + Namespace Admins - a namespace admin with Create + permissions can create or drop tables within that namespace, and take and restore + snapshots. A namespace admin with Admin permissions can perform + operations such as splits or major compactions on tables within that + namespace. - - - Column Family / Column Qualifier / Cell - - - Read: User can read at the specified scope. - - - Write: User can write at the specified scope. - - + Table Admins - A table admin can perform administrative + operations only on that table. A table admin with Create permissions + can create snapshots from that table or restore that table from a snapshot. A table + admin with Admin permissions can perform operations such as splits or + major compactions on that table. - - - Coprocessor Endpoint - Execute: the user can execute the coprocessor endpoint. + Users - Users can read or write data, or both. Users can also + execute coprocessor endpoints, if given Executable permissions. - - - Global + + + Real-World Example of Access Levels + + + + Job Title + Scope + Permissions + Description + + + + + Senior Administrator + Global + Access, Create + Manages the cluster and gives access to Junior + Administrators. + + + Junior Administrator + Global + Create + Creates tables and gives access to Table + Administrators. + + + Table Administrator + Table + Access + Maintains a table from an operations point of view. + + + Data Analyst + Table + Read + Creates reports from HBase data. + + + Web Application + Table + Read, Write + Puts data into HBase and uses HBase data to perform + operations. + + + + + +
This table shows how real-world titles might map to HBase permissions in + a hypothetical company.
+ + ACL Matrix + For more details on how ACLs map to specific HBase operations and tasks, see . + +
+
+ Implementation Details + Cell-level ACLs are implemented using tags (see ). In + order to use cell-level ACLs, you must be using HFile v3 and HBase 0.98 or newer. + + ACL Implementation Caveats - Superusers are specified as a comma-separated list of users and groups, in the - option in hbase-site.xml. - The superuser is equivalent to the root user in a UNIX - environment. As a minimum, the superuser should include the principal used to run - the HMaster process. Global admin privileges, which are implicitly granted to the - superuser, are required to create namespaces, switch the balancer on and off, or - take other actions with global consequences. The superuser can also grant all - permissions to all resources. + Files created by HBase are owned by the operating system user running the HBase + process. To interact with HBase files, you should use the API or bulk load + facility. - - - - ACL Matrix - For more details on how ACLs map to specific HBase operations and tasks, see . - - Cell-level ACLs are implemented using tags (see ). In - order to use cell-level ACLs, you must be using HFile v3 and HBase 0.98 or newer. - - ACL Implementation Caveats - - Files created by HBase are owned by the operating system user running the HBase - process. To interact with HBase files, you should use the API or bulk load - facility. - - - HBase does not model "roles" internally in HBase. Instead, group names can be - granted permissions. This allows external modeling of roles via group membership. - Groups are created and manipulated externally to HBase, via the Hadoop group mapping - service. - - -
-
- Server-Side Configuration - - - As a prerequisite, perform the steps in . - - Install and configure the AccessController coprocessor, by setting the following - properties in hbase-site.xml. These properties take a list of - classes. - - If you use the AccessController along with the VisibilityController, the - AccessController must come first in the list, because with both components active, - the VisibilityController will delegate access control on its system tables to the - AccessController. For an example of using both together, see . - + HBase does not model "roles" internally in HBase. Instead, group names can be + granted permissions. This allows external modeling of roles via group membership. + Groups are created and manipulated externally to HBase, via the Hadoop group mapping + service. + + +
+
+ Server-Side Configuration + + + As a prerequisite, perform the steps in . + + + Install and configure the AccessController coprocessor, by setting the following + properties in hbase-site.xml. These properties take a list of + classes. + + If you use the AccessController along with the VisibilityController, the + AccessController must come first in the list, because with both components active, + the VisibilityController will delegate access control on its system tables to the + AccessController. For an example of using both together, see . + + hbase.coprocessor.region.classes org.apache.hadoop.hbase.security.access.AccessController, org.apache.hadoop.hbase.security.token.TokenProvider @@ -817,15 +816,15 @@ grant 'rest_server', 'RWCA' true ]]> - Optionally, you can enable transport security, by setting - to auth-conf. This requires - HBase 0.98.4 or newer. - - - Set up the Hadoop group mapper in the Hadoop namenode's - core-site.xml. This is a Hadoop file, not an HBase file. - Customize it to your site's needs. Following is an example. - Optionally, you can enable transport security, by setting + to auth-conf. This + requires HBase 0.98.4 or newer. + + + Set up the Hadoop group mapper in the Hadoop namenode's + core-site.xml. This is a Hadoop file, not an HBase file. + Customize it to your site's needs. Following is an example. + hadoop.security.group.mapping org.apache.hadoop.security.LdapGroupsMapping @@ -871,140 +870,140 @@ grant 'rest_server', 'RWCA' cn ]]> - - - Optionally, enable the early-out evaluation strategy. Prior to HBase 0.98.0, if a - user was not granted access to a column family, or at least a column qualifier, an - AccessDeniedException would be thrown. HBase 0.98.0 removed this exception in order to - allow cell-level exceptional grants. To restore the old behavior in HBase - 0.98.0-0.98.6, set to - true in hbase-site.xml. In HBase 0.98.6, the - default has been returned to true. - - - Distribute your configuration and restart your cluster for changes to take - effect. - - - To test your configuration, log into HBase Shell as a given user and use the - whoami command to report the groups your user is part of. In this - example, the user is reported as being a member of the services - group. - + + + Optionally, enable the early-out evaluation strategy. Prior to HBase 0.98.0, if + a user was not granted access to a column family, or at least a column qualifier, an + AccessDeniedException would be thrown. HBase 0.98.0 removed this exception in order + to allow cell-level exceptional grants. To restore the old behavior in HBase + 0.98.0-0.98.6, set to + true in hbase-site.xml. In HBase 0.98.6, + the default has been returned to true. + + + Distribute your configuration and restart your cluster for changes to take + effect. + + + To test your configuration, log into HBase Shell as a given user and use the + whoami command to report the groups your user is part of. In + this example, the user is reported as being a member of the services + group. + hbase> whoami service (auth:KERBEROS) groups: services - - -
-
- Administration - Administration tasks can be performed from HBase Shell or via an API. - - API Examples - Many of the API examples below are taken from source files - hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java - and - hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java. - Neither the examples, nor the source files they are taken from, are part of the - public HBase API, and are provided for illustration only. Refer to the - official API for usage instructions. - - - - User and Group Administration - Users and groups are maintained external to HBase, in your directory. - - - Granting Access To A Namespace, Table, Column Family, or Cell - There are a few different types of syntax for grant statements. The first, and - most familiar, is as follows, with the table and column family being optional: - grant 'user', 'RWXCA', 'TABLE', 'CF', 'CQ' - Groups and users are granted access in the same way, but groups are prefixed with - an @ symbol. In the same way, tables and namespaces are specified - in the same way, but namespaces are prefixed with an @ - symbol. - It is also possible to grant multiple permissions against the same resource in a - single statement, as in this example. The first sub-clause maps users to - ACLs and the second sub-clause specifies the resource. - - HBase Shell support for granting and revoking access at the cell level is for - testing and verification support, and should not be employed for production use - because it won't apply the permissions to cells that don't exist yet. The correct - way to apply cell level permissions is to do so in the application code when storing - the values. - - - ACL Granularity and Evaluation Order - ACLs are evaluated from least granular to most granular, and when an ACL is - reached that grants permission, evaluation stops. This means that cell ACLs do not - override ACLs at less granularity. - - - HBase Shell - - - Global: - hbase> grant '@admins', 'RWXCA' - - - Namespace: - hbase> grant 'service', 'RWXCA', '@test-NS' - - - Table: - hbase> grant 'service', 'RWXCA', 'user' - - - Column Family: - hbase> grant '@developers', 'RW', 'user', 'i' - - - Column Qualifier: - hbase> grant 'service, 'RW', 'user', 'i', 'foo' - - - Cell: - The syntax for granting cell ACLs uses the following syntax: - grant <table>, \ + + +
+
+ Administration + Administration tasks can be performed from HBase Shell or via an API. + + API Examples + Many of the API examples below are taken from source files + hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/TestAccessController.java + and + hbase-server/src/test/java/org/apache/hadoop/hbase/security/access/SecureTestUtil.java. + Neither the examples, nor the source files they are taken from, are part of the + public HBase API, and are provided for illustration only. Refer to the official API + for usage instructions. + + + + User and Group Administration + Users and groups are maintained external to HBase, in your directory. + + + Granting Access To A Namespace, Table, Column Family, or Cell + There are a few different types of syntax for grant statements. The first, and + most familiar, is as follows, with the table and column family being + optional: + grant 'user', 'RWXCA', 'TABLE', 'CF', 'CQ' + Groups and users are granted access in the same way, but groups are prefixed + with an @ symbol. In the same way, tables and namespaces are + specified in the same way, but namespaces are prefixed with an @ + symbol. + It is also possible to grant multiple permissions against the same resource in a + single statement, as in this example. The first sub-clause maps users to ACLs and + the second sub-clause specifies the resource. + + HBase Shell support for granting and revoking access at the cell level is for + testing and verification support, and should not be employed for production use + because it won't apply the permissions to cells that don't exist yet. The correct + way to apply cell level permissions is to do so in the application code when + storing the values. + + + ACL Granularity and Evaluation Order + ACLs are evaluated from least granular to most granular, and when an ACL is + reached that grants permission, evaluation stops. This means that cell ACLs do not + override ACLs at less granularity. + + + HBase Shell + + + Global: + hbase> grant '@admins', 'RWXCA' + + + Namespace: + hbase> grant 'service', 'RWXCA', '@test-NS' + + + Table: + hbase> grant 'service', 'RWXCA', 'user' + + + Column Family: + hbase> grant '@developers', 'RW', 'user', 'i' + + + Column Qualifier: + hbase> grant 'service, 'RW', 'user', 'i', 'foo' + + + Cell: + The syntax for granting cell ACLs uses the following syntax: + grant <table>, \ { '<user-or-group>' => \ '<permissions>', ... }, \ { <scanner-specification> } - - - <user-or-group> is the user or group - name, prefixed with @ in the case of a group. - - - <permissions> is a string containing - any or all of "RWXCA", though only R and W are meaningful at cell - scope. - - - <scanner-specification> is the scanner - specification syntax and conventions used by the 'scan' shell command. For - some examples of scanner specifications, issue the following HBase Shell - command. - hbase> help "scan" - - - This example grants read access to the 'testuser' user and read/write access - to the 'developers' group, on cells in the 'pii' column which match the - filter. - hbase> grant 'user', \ + + + <user-or-group> is the user or group + name, prefixed with @ in the case of a group. + + + <permissions> is a string containing + any or all of "RWXCA", though only R and W are meaningful at cell + scope. + + + <scanner-specification> is the + scanner specification syntax and conventions used by the 'scan' shell + command. For some examples of scanner specifications, issue the following + HBase Shell command. + hbase> help "scan" + + + This example grants read access to the 'testuser' user and read/write + access to the 'developers' group, on cells in the 'pii' column which match the + filter. + hbase> grant 'user', \ { '@developers' => 'RW', 'testuser' => 'R' }, \ { COLUMNS => 'pii', FILTER => "(PrefixFilter ('test'))" } - The shell will run a scanner with the given criteria, rewrite the found - cells with new ACLs, and store them back to their exact coordinates. - - - - - API - The following example shows how to grant access at the - table level. - The shell will run a scanner with the given criteria, rewrite the found + cells with new ACLs, and store them back to their exact coordinates. + + + + + API + The following example shows how to grant access at the table level. + - To grant permissions at the cell level, you can use the - Mutation.setACL method: - To grant permissions at the cell level, you can use the + Mutation.setACL method: + perms) ]]> - Specifically, this example provides read permission to a user called - user1 on any cells contained in a particular Put - operation: - Specifically, this example provides read permission to a user called + user1 on any cells contained in a particular Put + operation: + - - - - Revoking Access Control From a Namespace, Table, Column Family, or Cell - The revoke command and API are twins of the grant command and - API, and the syntax is exactly the same. The only exception is that you cannot revoke - permissions at the cell level. You can only revoke access that has previously been - granted, and a revoke statement is not the same thing as explicit - denial to a resource. - - HBase Shell support for granting and revoking access is for testing and verification - support, and should not be employed for production use because it won't apply the - permissions to cells that don't exist yet. The correct way to apply cell-level - permissions is to do so in the application code when storing the values. - - - Revoking Access To a Table - + + + + Revoking Access Control From a Namespace, Table, Column Family, or Cell + The revoke command and API are twins of the grant command and + API, and the syntax is exactly the same. The only exception is that you cannot + revoke permissions at the cell level. You can only revoke access that has previously + been granted, and a revoke statement is not the same thing as + explicit denial to a resource. + + HBase Shell support for granting and revoking access is for testing and + verification support, and should not be employed for production use because it + won't apply the permissions to cells that don't exist yet. The correct way to + apply cell-level permissions is to do so in the application code when storing the + values. + + + Revoking Access To a Table + - - - - Showing a User's Effective Permissions - - HBase Shell - hbase> user_permission 'user' - hbase> user_permission '.*' - hbase> user_permission JAVA_REGEX - - - API - + + + Showing a User's Effective Permissions + + HBase Shell + hbase> user_permission 'user' + hbase> user_permission '.*' + hbase> user_permission JAVA_REGEX + + + API + - - - + + + +