Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Security overview documentation (#10339) 

* initial file

* initial file

* security overview added

* ldap added

* spacing adjustments

* nits

* security graphics and doc review

* Update docs/operations/security-overview.md

Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>

* Update docs/operations/security-user-auth.md

Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>

* Update docs/operations/security-overview.md

Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>

* Update docs/operations/security-overview.md

Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>

* updates frm review

* review comments

* finish up review and light edits

* broken links

* spell check

Co-authored-by: Jonathan Wei <jon-wei@users.noreply.github.com>
This commit is contained in:
sthetland 2020-11-19 15:24:58 -08:00 committed by GitHub
parent 2f4d6da33f
commit ba915b7f56
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
12 changed files with 677 additions and 127 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
docs/assets/security-model-1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 83 KiB

BIN
docs/assets/security-model-2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

101
docs/development/extensions-core/druid-basic-security.md
View File

@ -450,108 +450,9 @@ Content: List of JSON Resource-Action objects, e.g.:
The "name" field for resources in the permission definitions are regexes used to match resource names during authorization checks.
Please see [Defining permissions](#defining-permissions) for more details.
Please see [Defining permissions](../../operations/security-user-auth.md#defining-permissions) for more details.
##### Cache Load Status
`GET(/druid-ext/basic-security/authorization/loadStatus)`
Return the current load status of the local caches of the authorization Druid metadata store.
## Default user accounts
### Authenticator
If `druid.auth.authenticator.<authenticator-name>.initialAdminPassword` is set, a default admin user named "admin" will be created, with the specified initial password. If this configuration is omitted, the "admin" user will not be created.
If `druid.auth.authenticator.<authenticator-name>.initialInternalClientPassword` is set, a default internal system user named "druid_system" will be created, with the specified initial password. If this configuration is omitted, the "druid_system" user will not be created.
### Authorizer
Each Authorizer will always have a default "admin" and "druid_system" user with full privileges.
## Defining permissions
There are two action types in Druid: READ and WRITE
There are three resource types in Druid: DATASOURCE, CONFIG, and STATE.
### DATASOURCE
Resource names for this type are datasource names. Specifying a datasource permission allows the administrator to grant users access to specific datasources.
### CONFIG
There are two possible resource names for the "CONFIG" resource type, "CONFIG" and "security". Granting a user access to CONFIG resources allows them to access the following endpoints.
"CONFIG" resource name covers the following endpoints:
|Endpoint|Process Type|
|--------|---------|
|`/druid/coordinator/v1/config`|coordinator|
|`/druid/indexer/v1/worker`|overlord|
|`/druid/indexer/v1/worker/history`|overlord|
|`/druid/worker/v1/disable`|middleManager|
|`/druid/worker/v1/enable`|middleManager|
"security" resource name covers the following endpoint:
|Endpoint|Process Type|
|--------|---------|
|`/druid-ext/basic-security/authentication`|coordinator|
|`/druid-ext/basic-security/authorization`|coordinator|
### STATE
There is only one possible resource name for the "STATE" config resource type, "STATE". Granting a user access to STATE resources allows them to access the following endpoints.
"STATE" resource name covers the following endpoints:
|Endpoint|Process Type|
|--------|---------|
|`/druid/coordinator/v1`|coordinator|
|`/druid/coordinator/v1/rules`|coordinator|
|`/druid/coordinator/v1/rules/history`|coordinator|
|`/druid/coordinator/v1/servers`|coordinator|
|`/druid/coordinator/v1/tiers`|coordinator|
|`/druid/broker/v1`|broker|
|`/druid/v2/candidates`|broker|
|`/druid/indexer/v1/leader`|overlord|
|`/druid/indexer/v1/isLeader`|overlord|
|`/druid/indexer/v1/action`|overlord|
|`/druid/indexer/v1/workers`|overlord|
|`/druid/indexer/v1/scaling`|overlord|
|`/druid/worker/v1/enabled`|middleManager|
|`/druid/worker/v1/tasks`|middleManager|
|`/druid/worker/v1/task/{taskid}/shutdown`|middleManager|
|`/druid/worker/v1/task/{taskid}/log`|middleManager|
|`/druid/historical/v1`|historical|
|`/druid-internal/v1/segments/`|historical|
|`/druid-internal/v1/segments/`|peon|
|`/druid-internal/v1/segments/`|realtime|
|`/status`|all process types|
### HTTP methods
For information on what HTTP methods are supported on a particular request endpoint, please refer to the [API documentation](../../operations/api-reference.md).
GET requires READ permission, while POST and DELETE require WRITE permission.
### SQL Permissions
Queries on Druid datasources require DATASOURCE READ permissions for the specified datasource.
Queries on the [INFORMATION_SCHEMA tables](../../querying/sql.html#information-schema) will
return information about datasources that the caller has DATASOURCE READ access to. Other
datasources will be omitted.
Queries on the [system schema tables](../../querying/sql.html#system-schema) require the following permissions:
- `segments`: Segments will be filtered based on DATASOURCE READ permissions.
- `servers`: The user requires STATE READ permissions.
- `server_segments`: The user requires STATE READ permissions and segments will be filtered based on DATASOURCE READ permissions.
- `tasks`: Tasks will be filtered based on DATASOURCE READ permissions.
## Configuration Propagation
To prevent excessive load on the Coordinator, the Authenticator and Authorizer user/role Druid metadata store state is cached on each Druid process.
Each process will periodically poll the Coordinator for the latest Druid metadata store state, controlled by the `druid.auth.basic.common.pollingPeriod` and `druid.auth.basic.common.maxRandomDelay` properties.
When a configuration update occurs, the Coordinator can optionally notify each process with the updated Druid metadata store state. This behavior is controlled by the `enableCacheNotifications` and `cacheNotificationTimeout` properties on Authenticators and Authorizers.
Note that because of the caching, changes made to the user/role Druid metadata store may not be immediately reflected at each Druid process.

196
docs/operations/auth-ldap.md Normal file
View File

@ -0,0 +1,196 @@
---
id: auth-ldap
title: "LDAP auth"
---
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
This page describes how to set up Druid user authentication and authorization through LDAP. The first step is to enable LDAP authentication and authorization for Druid. You then map an LDAP group to roles and assign permissions and users to roles.
## Enable LDAP in Druid
Before starting, verify that the active directory is reachable from the Druid Master servers. Command line tools such as `ldapsearch` and `ldapwhoami`, which are included with OpenLDAP, are useful for this testing. 
### Check the connection
First test that the basic connection and user credential works. For example, given a user `uuser1@example.com`, try:
```bash
ldapwhoami -vv -H ldap://<ip_address>:389  -D"uuser1@example.com" -W
```
Enter the password associated with the user when prompted and verify that the command succeeded. If it didn't, try the following troubleshooting steps:
* Verify that you've used the correct port for your LDAP instance. By default, the LDAP port is 389, but double-check with your LDAP admin if unable to connect.
* Check whether a network firewall is not preventing connections to the LDAP port.
* Check whether LDAP clients need to be specifically whitelisted at the LDAP server to be able to reach it. If so, add the Druid Coordinator server to the AD whitelist.
### Check the search criteria
After verifying basic connectivity, check your search criteria. For example, the command for searching for user `uuser1@example.com ` is as follows:
```bash
ldapsearch -x -W -H ldap://<ldap_server>  -D"uuser1@example.com" -b "dc=example,dc=com" "(sAMAccountName=uuser1)"
```
Note the `memberOf` attribute in the results; it shows the groups that the user belongs to. You will use this value to map the LDAP group to the Druid roles later. The sAMAccountName attribute contains the authenticated user identity.
## Configure Druid user authentication with LDAP/Active Directory 
1. Enable the `druid-basic-security` extension in the `common.runtime.properties` file. See [Security Overview](security-overview.md) for details.
2. As a best practice, create a user in LDAP to be used for internal communication with Druid.
3. In `common.runtime.properties`, update LDAP-related properties, as shown in the following listing: 
```
druid.auth.authenticatorChain=["ldap"]
druid.auth.authenticator.ldap.type=basic
druid.auth.authenticator.ldap.enableCacheNotifications=true
druid.auth.authenticator.ldap.credentialsValidator.type=ldap
druid.auth.authenticator.ldap.credentialsValidator.url=ldap://<AD host>:<AD port>
druid.auth.authenticator.ldap.credentialsValidator.bindUser=<AD admin user, e.g.: Administrator@example.com>
druid.auth.authenticator.ldap.credentialsValidator.bindPassword=<AD admin password>
druid.auth.authenticator.ldap.credentialsValidator.baseDn=<base dn, e.g.: dc=example,dc=com>
druid.auth.authenticator.ldap.credentialsValidator.userSearch=<The LDAP search, e.g.: (&(sAMAccountName=%s)(objectClass=user))>
druid.auth.authenticator.ldap.credentialsValidator.userAttribute=sAMAccountName
druid.auth.authenticator.ldap.authorizerName=ldapauth
druid.escalator.type=basic
druid.escalator.internalClientUsername=<AD internal user, e.g.: internal@example.com>
druid.escalator.internalClientPassword=Welcome123
druid.escalator.authorizerName=ldapauth
druid.auth.authorizers=["ldapauth"]
druid.auth.authorizer.ldapauth.type=basic
druid.auth.authorizer.ldapauth.initialAdminUser=AD user who acts as the initial admin user, e.g.: internal@example.com>
druid.auth.authorizer.ldapauth.initialAdminRole=admin
druid.auth.authorizer.ldapauth.roleProvider.type=ldap
```
Notice that the LDAP user created in the previous step, `internal@example.com`, serves as the internal client user and the initial admin user.
## Use LDAP groups to assign roles
You can map LDAP groups to a role in Druid. Members in the group get access to the permissions of the corresponding role. 
### Step 1: Create a role
First create the role in Druid using the Druid REST API.
Creating a role involves submitting a POST request to the Coordinator process. 
The following REST APIs to create the role to read access for datasource, config, state.
> As mentioned, the REST API calls need to address the Coordinator node. The examples used below use localhost as the Coordinator host and 8081 as the port. Adjust these settings according to your deployment.
Call the following API to create role `readRole` . 
```
curl -i -v  -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/roles/readRole
```
Check that the role has been created successfully by entering the following:
```
curl -i -v  -H "Content-Type: application/json" -u internal -X GET  http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/roles
```
### Step 2: Add permissions to a role 
You can now add one or more permission to the role. The following example adds read-only access to a `wikipedia` data source.  
Given the following JSON in a file named `perm.json`:
```
[{ "resource": { "name": "wikipedia", "type": "DATASOURCE" }, "action": "READ" }
,{ "resource": { "name": ".*", "type": "STATE" }, "action": "READ" },
{ "resource": {"name": ".*", "type": "CONFIG"}, "action": "READ"}]
```
The following command associates the permissions in the JSON file with the role
```
curl -i -v  -H "Content-Type: application/json" -u internal -X POST -d@perm.json  http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/roles/readRole/permissions
```
Note that the STATE and CONFIG permissions in `perm.json` are needed to see the data source in the Druid console. If only querying permissions are needed, the READ action is sufficient:
```
[{ "resource": { "name": "wikipedia", "type": "DATASOURCE" }, "action": "READ" }]
```
You can also provide the name in the form of regular expression. For example, to give access to all data sources starting with `wiki`, specify the name as  `{ "name": "wiki.*", .....`.
### Step 3: Create group Mapping 
The following shows an example of a group to role mapping. It assumes that a group named `group1` exists in the directory. Also assuming the following role mapping in a file named `groupmap.json`:
```
{
    "name": "group1map",
    "groupPattern": "CN=group1,CN=Users,DC=example,DC=com",
    "roles": [
        "readRole"
    ]
}
```
You can configure the mapping as follows:
```
curl -i -v  -H "Content-Type: application/json" -u internal -X POST -d @groupmap.json http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1map
```
To check whether the group mapping was created successfully, run the following command:
```
curl -i -v  -H "Content-Type: application/json" -u internal -X GET http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings
```
To check the details of a specific group mapping, use the following:
```
curl -i -v  -H "Content-Type: application/json" -u internal -X GET http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1map
```
To add additional roles to the group mapping, use the following API:
```
curl -i -v  -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1/roles/<newrole> 
```
### Step 4. Assign roles for individual LDAP users
Once LDAP is enabled, only user passwords are verified with LDAP. You add the LDAP user to Druid as follows: 
```
curl -i -v  -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authentication/db/ldap/users/<AD user> 
```
### Step 5. Assign the role to the user
The following command shows how to assign a role to a user:
```
curl -i -v  -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/users/<AD user>/roles/<rolename> 
```
For more information about security and the basic security extension, see [Security Overview](security-overview.md).

21
docs/operations/password-provider.md
View File

@ -23,14 +23,15 @@ title: "Password providers"
-->
Apache Druid needs some passwords for accessing various secured systems like metadata store, Key Store containing server certificates etc.
All these passwords have corresponding runtime properties associated with them, for example `druid.metadata.storage.connector.password` corresponds to the metadata store password.
Passwords help secure Apache Druid systems such as the metadata store and the keystore that contains server certificates, and so on.
By default users can directly set the passwords in plaintext for these runtime properties, for example `druid.metadata.storage.connector.password=pwd` sets the metadata store password
to be used by Druid to connect to metadata store to `pwd`. Apart from this, users can use environment variables to get password in following way -
These passwords have corresponding runtime properties associated with them, for example `druid.metadata.storage.connector.password` corresponds to the metadata store password.
Environment variable password provider provides password by looking at specified environment variable. Use this in order to avoid specifying password in runtime.properties file.
e.g
By default users can directly set the passwords in plaintext for runtime properties. For example, `druid.metadata.storage.connector.password=pwd` sets the password to be used by Druid to connect to the metadata store to `pwd`. Alternatively, users can can set passwords as environment variables.
Environment variable passwords allow users to avoid exposing passwords in the `runtime.properties` file.
You can set an environment variable password as in the following example:
```json
druid.metadata.storage.connector.password={ "type": "environment", "variable": "METADATA_STORAGE_PASSWORD" }
@ -43,11 +44,11 @@ The values are described below.
|`type`|String|password provider type|Yes: `environment`|
|`variable`|String|environment variable to read password from|Yes|
However, many times users may want their own way to optionally securely fetch password during runtime of the Druid process.
Druid allows this by users to implement their own `PasswordProvider` interface and create a Druid extension to register this implementation at Druid process startup.
Please have a look at "Adding a new Password Provider implementation" on this [page](../development/modules.md) to learn more.
Another option that provides even greater control is to securely fetch passwords at runtime using a custom extension of the `PasswordProvider` interface that is registered at Druid process startup.
To use this implementation, simply set the relevant password runtime property to something similar as was done for Environment variable password provider like -
For more information, see [Adding a new Password Provider implementation](../development/modules.md#adding-a-new-password-provider-implementation).
To use this implementation, simply set the relevant password runtime property similarly to how was shown for the environment variable password:
```json
druid.metadata.storage.connector.password={ "type": "<registered_password_provider_name>", "<jackson_property>": "<value>", ... }

265
docs/operations/security-overview.md Normal file
View File

@ -0,0 +1,265 @@
---
id: security-overview
title: "Security overview"
---
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
## Overview
By default, security features in Druid are disabled, which simplifies the initial deployment experience. However, security features must be configured in a production deployment. These features including TLS, authentication, and authorization.
To implement Druid security, you configure authenticators and authorizers. Authenticators control the way user identities are verified, while authorizers map the authenticated users (via user roles) to the datasources they are permitted to access. Consequently, implementing Druid security also involves consideration of your datasource scheme, given they represent the granularity at which data access permissions are allocated.
The following graphic depicts the course of request through the authentication process:
![Druid security check flow](../assets/security-model-1.png "Druid security check flow")
This document gives you an overview of security features in Druid and how to configure them, and some best practices for securing Druid.
## Best practices
* Do not expose the Druid Console without authentication on untrusted networks. Access to the console effectively confers access the file system on the installation machine, via file browsers in the UI. You should use an API gateway that restricts who can connect from untrusted networks, whitelist the specific APIs that your users need to access, and implements account lockout and throttling features.
* Grant users the minimum permissions necessary to perform their functions. For instance, do not allow user who only need to query data to write to data sources or view state.
* Disable JavaScript, as noted in the [Security section](https://druid.apache.org/docs/latest/development/javascript.html#security) of the JavaScript guide.
* Run Druid as an unprivileged Unix user on the installation machine (not root).
> This is an important point! Administrator users on Druid have the same permission as the Unix user account it is running under. If the Druid process is running under the root user account in the OS, then Administrator users on Druid can read/write all files that the root account has access to, including sensitive files such as `/etc/passwd`.
You can configure authentication and authorization to control access to the the Druid APIs. The first step is enabling TLS for the cluster nodes. Then configure users, roles, and permissions, as described in the following sections.
The configuration settings mentioned below are primarily located in the `common.runtime.properties` file. Note that you need to make the configuration changes on each Druid server in the cluster.
## Enable TLS
The first step in securing Druid is enabling TLS. You can enable TLS to secure external client connections to Druid as well as connections between cluster nodes.
The configuration steps are:
1. Enable TLS by adding `druid.enableTlsPort=true` to `common.runtime.properties` on each node in the Druid cluster.
2. Disable the non-TLS port by setting `druid.enablePlaintextPort` to `false`.
2. Follow the steps in [Understanding Certificates and Keys](https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#understanding-certificates-and-keys) to generate or import a key and certificate.
3. Configure the keystore and truststore settings in `common.runtime.properties`. The file should look something like this:
```
druid.enablePlaintextPort=false
druid.enableTlsPort=true
druid.server.https.keyStoreType=jks
druid.server.https.keyStorePath=imply-keystore.jks
druid.server.https.keyStorePassword=secret123 # replace with your own password
druid.server.https.certAlias=druid
druid.client.https.protocol=TLSv1.2
druid.client.https.trustStoreType=jks
druid.client.https.trustStorePath=imply-truststore.jks
druid.client.https.trustStorePassword=secret123 # replace with your own password
```
4. Add the `simple-client-sslcontext` extension to `druid.extensions.loadList` in `common.runtime.properties`. This enables TLS for Druid nodes acting as clients.
5. Restart the cluster.
For more information, see [TLS support](tls-support.md) and [Simple SSLContext Provider Module](../development/extensions-core/simple-client-sslcontext.md).
Druid uses Jetty as its embedded web server. Therefore you refer to [Understanding Certificates and Keys](https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html) for complete instructions.
## Enable an authenticator
To authenticate requests in Druid, you configure an Authenticator. Authenticator extensions exist for HTTP basic authentication, LDAP, and Kerberos.
The following takes you through sample configuration steps for enabling basic auth:
1. Add the `druid-basic-security` extension to `druid.extensions.loadList` in `common.runtime.properties`. For the quickstart installation, for example, the properties file is at `conf/druid/cluster/_common`:
```
druid.extensions.loadList=["druid-basic-security", "druid-histogram", "druid-datasketches", "druid-kafka-indexing-service", "imply-utility-belt"]
```
2. Configure the basic Authenticator, Authorizer, and Escalator settings in the same common.runtime.properties file. For example:
```
# Druid basic security
druid.auth.authenticatorChain=["MyBasicMetadataAuthenticator"]
druid.auth.authenticator.MyBasicMetadataAuthenticator.type=basic
druid.auth.authenticator.MyBasicMetadataAuthenticator.initialAdminPassword=password1
druid.auth.authenticator.MyBasicMetadataAuthenticator.initialInternalClientPassword=password2
druid.auth.authenticator.MyBasicMetadataAuthenticator.credentialsValidator.type=metadata
druid.auth.authenticator.MyBasicMetadataAuthenticator.skipOnFailure=false
druid.auth.authenticator.MyBasicMetadataAuthenticator.authorizerName=MyBasicMetadataAuthorizer
# Escalator
druid.escalator.type=basic
druid.escalator.internalClientUsername=druid_system
druid.escalator.internalClientPassword=password2
druid.escalator.authorizerName=MyBasicMetadataAuthorizer
druid.auth.authorizers=["MyBasicMetadataAuthorizer"]
druid.auth.authorizer.MyBasicMetadataAuthorizer.type=basic
```
3. Restart the cluster.
See [Authentication and Authorization](../design/auth.md) for more information about the Authenticator, Escalator, and Authorizer concepts. See [Basic Security](../development/extensions-core/druid-basic-security.md) for more information about the extension used in the examples above, and [Kerberos](../development/extensions-core/druid-kerberos.md) for Kerberos authentication.
## Enable authorizers
After enabling the basic auth extension, you can add users, roles, and permissions via the Druid Coordinator `user` endpoint. Note that you cannot assign permissions directly to individual users. They must be assigned through roles.
The following diagram depicts the authorization model, and the relationship between users, roles, permissions, and resources.
![Druid Security model](../assets/security-model-2.png "Druid security model")
The following steps walk through a sample setup procedure:
> The default Coordinator API port is 8081 for non-TLS connections and 8281 for secured connections.
1. Create a user by issuing a POST request to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>`, replacing USERNAME with the new username. For example:
```
curl -u admin:password -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname
```
> If you have TLS enabled, be sure to adjust the curl command accordingly. For example, if your Druid servers use self-signed certificates, you may choose to include the `insecure` curl option to forgo certificate checking for the curl command.
2. Add a credential for the user by issuing a POST to `druid-ext/basic-security/authentication/db/MyBasicMetadataAuthenticator/users/<USERNAME>/credentials`. For example:
```
curl -u admin:password -H'Content-Type: application/json' -XPOST --data-binary @pass.json https://my-coordinator-ip:8281/druid-ext/basic-security/authentication/db/basic/users/myname/credentials
```
The password is conveyed in the `pass.json` file in the following form:
```
{
"password": "password"
}
```
2. For each authenticator user you create, create a corresponding authorizer user by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>`. For example:
```
curl -u admin:password -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname
```
3. Create authorizer roles to control permissions by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>`. For example:
```
curl -u admin:password -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole
```
4. Assign roles to users by issuing a POST request to `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/users/<USERNAME>/roles/<ROLENAME>`. For example:
```
curl -u admin:password -XPOST https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/users/myname/roles/myrole | jq
```
5. Finally, attach permissions to the roles to control how they can interact with Druid at `druid-ext/basic-security/authorization/db/MyBasicMetadataAuthorizer/roles/<ROLENAME>/permissions`.
For example:
```
curl -u admin:password -H'Content-Type: application/json' -XPOST --data-binary @perms.json https://my-coordinator-ip:8281/druid-ext/basic-security/authorization/db/basic/roles/myrole/permissions
```
The payload of `perms.json` should be in the form:
```
[
{
"resource": {
"name": "<PATTERN>",
"type": "DATASOURCE"
},
"action": "READ"
},
{
"resource": {
"name": "STATE",
"type": "STATE"
},
"action": "READ"
}
]
```
## Configuring an LDAP authenticator
As an alternative to using the basic metadata authenticator, as shown in the previous section, you can use LDAP to authenticate users. The following steps provide an overview of the setup steps. For more information on these settings, see [Properties for LDAP user authentication](../development/extensions-core/druid-basic-security.md#properties-for-ldap-user-authentication).
1. In `common.runtime.properties`, add LDAP to the authenticator chain in the order in which you want requests to be evaluated. For example:
```
# Druid basic security
druid.auth.authenticatorChain=["ldap", "MyBasicMetadataAuthenticator"]
```
2. Configure LDAP settings in `common.runtime.properties` as appropriate for your LDAP scheme and system. For example:
```
druid.auth.authenticator.ldap.type=basic
druid.auth.authenticator.ldap.enableCacheNotifications=true
druid.auth.authenticator.ldap.credentialsValidator.type=ldap
druid.auth.authenticator.ldap.credentialsValidator.url=ldap://ad_host:389
druid.auth.authenticator.ldap.credentialsValidator.bindUser=ad_admin_user
druid.auth.authenticator.ldap.credentialsValidator.bindPassword=ad_admin_password
druid.auth.authenticator.ldap.credentialsValidator.baseDn=dc=example,dc=com
druid.auth.authenticator.ldap.credentialsValidator.userSearch=(&(sAMAccountName=%s)(objectClass=user))
druid.auth.authenticator.ldap.credentialsValidator.userAttribute=sAMAccountName
druid.auth.authenticator.ldap.authorizerName=ldapauth
druid.escalator.type=basic
druid.escalator.internalClientUsername=ad_interal_user
druid.escalator.internalClientPassword=Welcome123
druid.escalator.authorizerName=ldapauth
druid.auth.authorizers=["ldapauth"]
druid.auth.authorizer.ldapauth.type=basic
druid.auth.authorizer.ldapauth.initialAdminUser=<ad_initial_admin_user>
druid.auth.authorizer.ldapauth.initialAdminRole=admin
druid.auth.authorizer.ldapauth.roleProvider.type=ldap
```
3. Use the Druid API to create the group mapping and allocate initial roles. For example, using curl and given a group named `group1` in the directory, run:
```
curl -i -v -H "Content-Type: application/json" -u internal -X POST -d @groupmap.json http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1map
```
The `groupmap.json` file contents would be something like:
```
{
"name": "group1map",
"groupPattern": "CN=group1,CN=Users,DC=example,DC=com",
"roles": [
"readRole"
]
}
```
4. Check if the group mapping is created successfully by executing the following API. This lists all group mappings.
```
curl -i -v -H "Content-Type: application/json" -u internal -X GET http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings
```
Alternatively, to check the details of a specific group mapping, use the following API:
```
curl -i -v -H "Content-Type: application/json" -u internal -X GET http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1map
```
5. To add additional roles to the group mapping, use the following API:
```
curl -i -v -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/groupMappings/group1/roles/<newrole>
```
6. Add the LDAP user to Druid. To add a user, use the following authentication API:
```
curl -i -v -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authentication/db/ldap/users/<ad_user>
```
7. Use the following command to assign the role to a user:
```
curl -i -v -H "Content-Type: application/json" -u internal -X POST http://localhost:8081/druid-ext/basic-security/authorization/db/ldapauth/users/<ad_user>/roles/<rolename>
```
Congratulations, you have configured permissions for user-assigned roles in Druid!

151
docs/operations/security-user-auth.md Normal file
View File

@ -0,0 +1,151 @@
---
id: security-user-auth
title: "User authentication and authorization"
---
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
This document describes the Druid security model that extensions use to enable user authentication and authorization services to Druid.
## Authentication and authorization model
At the center of the Druid user authentication and authorization model are _resources_ and _actions_. A resource is something that authenticated users are trying to access or modify. An action is something that users are trying to do.
There are three resource types:
* DATASOURCE &ndash; Each Druid table (i.e., `tables` in the `druid` schema in SQL) is a resource.
* CONFIG &ndash; Configuration resources exposed by the cluster components.
* STATE &ndash; Cluster-wide state resources.
For specific resources associated with the types, see the endpoint list below and corresponding descriptions in [API Reference](./api-reference.md).
There are two actions:
* READ &ndash; Used for read-only operations.
* WRITE &ndash; Used for operations that are not read-only.
In practice, most deployments will only need to define two classes of users:
* Admins, who have WRITE action permissions on all resource types. These users will add datasources and administer the system.
* Data users, who only need READ access to DATASOURCE. These users should access Query APIs only through an API gateway. Other APIs and permissions include functionality that should be limited to server admins.
It is important to note that WRITE access to DATASOURCE grants a user broad access. For instance, such users will have access to the Druid file system, S3 buckets, and credentials, among other things. As such, the ability to add and manage datasources should be allocated selectively to administrators.
## Default user accounts
### Authenticator
If `druid.auth.authenticator.<authenticator-name>.initialAdminPassword` is set, a default admin user named "admin" will be created, with the specified initial password. If this configuration is omitted, the "admin" user will not be created.
If `druid.auth.authenticator.<authenticator-name>.initialInternalClientPassword` is set, a default internal system user named "druid_system" will be created, with the specified initial password. If this configuration is omitted, the "druid_system" user will not be created.
### Authorizer
Each Authorizer will always have a default "admin" and "druid_system" user with full privileges.
## Defining permissions
There are two action types in Druid: READ and WRITE
There are three resource types in Druid: DATASOURCE, CONFIG, and STATE.
### DATASOURCE
Resource names for this type are datasource names. Specifying a datasource permission allows the administrator to grant users access to specific datasources.
### CONFIG
There are two possible resource names for the "CONFIG" resource type, "CONFIG" and "security". Granting a user access to CONFIG resources allows them to access the following endpoints.
"CONFIG" resource name covers the following endpoints:
|Endpoint|Process Type|
|--------|---------|
|`/druid/coordinator/v1/config`|coordinator|
|`/druid/indexer/v1/worker`|overlord|
|`/druid/indexer/v1/worker/history`|overlord|
|`/druid/worker/v1/disable`|middleManager|
|`/druid/worker/v1/enable`|middleManager|
"security" resource name covers the following endpoint:
|Endpoint|Process Type|
|--------|---------|
|`/druid-ext/basic-security/authentication`|coordinator|
|`/druid-ext/basic-security/authorization`|coordinator|
### STATE
There is only one possible resource name for the "STATE" config resource type, "STATE". Granting a user access to STATE resources allows them to access the following endpoints.
"STATE" resource name covers the following endpoints:
|Endpoint|Process Type|
|--------|---------|
|`/druid/coordinator/v1`|coordinator|
|`/druid/coordinator/v1/rules`|coordinator|
|`/druid/coordinator/v1/rules/history`|coordinator|
|`/druid/coordinator/v1/servers`|coordinator|
|`/druid/coordinator/v1/tiers`|coordinator|
|`/druid/broker/v1`|broker|
|`/druid/v2/candidates`|broker|
|`/druid/indexer/v1/leader`|overlord|
|`/druid/indexer/v1/isLeader`|overlord|
|`/druid/indexer/v1/action`|overlord|
|`/druid/indexer/v1/workers`|overlord|
|`/druid/indexer/v1/scaling`|overlord|
|`/druid/worker/v1/enabled`|middleManager|
|`/druid/worker/v1/tasks`|middleManager|
|`/druid/worker/v1/task/{taskid}/shutdown`|middleManager|
|`/druid/worker/v1/task/{taskid}/log`|middleManager|
|`/druid/historical/v1`|historical|
|`/druid-internal/v1/segments/`|historical|
|`/druid-internal/v1/segments/`|peon|
|`/druid-internal/v1/segments/`|realtime|
|`/status`|all process types|
### HTTP methods
For information on what HTTP methods are supported on a particular request endpoint, please refer to the [API documentation](./api-reference.md).
GET requires READ permission, while POST and DELETE require WRITE permission.
### SQL Permissions
Queries on Druid datasources require DATASOURCE READ permissions for the specified datasource.
Queries on the [INFORMATION_SCHEMA tables](../querying/sql.md#information-schema) will
return information about datasources that the caller has DATASOURCE READ access to. Other
datasources will be omitted.
Queries on the [system schema tables](../querying/sql.md#system-schema) require the following permissions:
- `segments`: Segments will be filtered based on DATASOURCE READ permissions.
- `servers`: The user requires STATE READ permissions.
- `server_segments`: The user requires STATE READ permissions and segments will be filtered based on DATASOURCE READ permissions.
- `tasks`: Tasks will be filtered based on DATASOURCE READ permissions.
## Configuration Propagation
To prevent excessive load on the Coordinator, the Authenticator and Authorizer user/role Druid metadata store state is cached on each Druid process.
Each process will periodically poll the Coordinator for the latest Druid metadata store state, controlled by the `druid.auth.basic.common.pollingPeriod` and `druid.auth.basic.common.maxRandomDelay` properties.
When a configuration update occurs, the Coordinator can optionally notify each process with the updated Druid metadata store state. This behavior is controlled by the `enableCacheNotifications` and `cacheNotificationTimeout` properties on Authenticators and Authorizers.
Note that because of the caching, changes made to the user/role Druid metadata store may not be immediately reflected at each Druid process.

17
docs/operations/tls-support.md
View File

@ -29,17 +29,20 @@ title: "TLS support"
|`druid.enablePlaintextPort`|Enable/Disable HTTP connector.|`true`|
|`druid.enableTlsPort`|Enable/Disable HTTPS connector.|`false`|
Although not recommended but both HTTP and HTTPS connectors can be enabled at a time and respective ports are configurable using `druid.plaintextPort`
Although not recommended, the HTTP and HTTPS connectors can both be enabled at a time. The respective ports are configurable using `druid.plaintextPort`
and `druid.tlsPort` properties on each process. Please see `Configuration` section of individual processes to check the valid and default values for these ports.
## Jetty server configuration
Apache Druid uses Jetty as an embedded web server. To get familiar with TLS/SSL in general and related concepts like Certificates etc.
reading this [Jetty documentation](http://www.eclipse.org/jetty/documentation/9.4.x/configuring-ssl.html) might be helpful.
To get more in depth knowledge of TLS/SSL support in Java in general, please refer to this [guide](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html).
The documentation [here](http://www.eclipse.org/jetty/documentation/9.4.x/configuring-ssl.html#configuring-sslcontextfactory)
can help in understanding TLS/SSL configurations listed below. This [document](http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html) lists all the possible
values for the below mentioned configs among others provided by Java implementation.
Apache Druid uses Jetty as its embedded web server.
To get familiar with TLS/SSL, along with related concepts like keys and certificates,
read [Configuring SSL/TLS](https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html) in the Jetty documentation.
To get more in-depth knowledge of TLS/SSL support in Java in general, refer to the [Java Secure Socket Extension (JSSE) Reference Guide](http://docs.oracle.com/javase/8/docs/technotes/guides/security/jsse/JSSERefGuide.html).
The [Configuring the Jetty SslContextFactory](https://www.eclipse.org/jetty/documentation/current/configuring-ssl.html#configuring-sslcontextfactory)
section can help in understanding TLS/SSL configurations listed below. Finally, [Java Cryptography Architecture
Standard Algorithm Name Documentation for JDK 8](http://docs.oracle.com/javase/8/docs/technotes/guides/security/StandardNames.html) lists all possible
values for the configs belong, among others provided by Java implementation.
|Property|Description|Default|Required|
|--------|-----------|-------|--------|

2
docs/querying/sql.md
View File

@ -1221,5 +1221,5 @@ Druid SQL planning occurs on the Broker and is configured by
## Security
Please see [Defining SQL permissions](../development/extensions-core/druid-basic-security.html#sql-permissions) in the
Please see [Defining SQL permissions](../operations/security-user-auth.md#sql-permissions) in the
basic security documentation for information on what permissions are needed for making SQL queries.

4
website/.spelling
View File

@ -114,6 +114,7 @@ JRE
JS
JSON
JsonPath
JSSE
JVM
JVMs
Joda
@ -141,6 +142,7 @@ OCF
OLAP
OOMs
OpenJDK
OpenLDAP
OpenTSDB
OutputStream
ParAccel
@ -164,6 +166,7 @@ Samza
Splunk
SqlFirehose
SqlParameter
SslContextFactory
StatsD
TCP
TGT
@ -404,6 +407,7 @@ unmerged
unparseable
unparsed
unsetting
untrusted
useFilterCNF
uptime
uris

12
website/i18n/en.json
View File

@ -23,6 +23,9 @@
"comparisons/druid-vs-sql-on-hadoop": {
"title": "Apache Druid vs SQL-on-Hadoop"
},
"configuration/human-readable-byte": {
"title": "Human-readable Byte Configuration Reference"
},
"configuration/index": {
"title": "Configuration reference"
},
@ -306,6 +309,9 @@
"operations/api-reference": {
"title": "API reference"
},
"operations/auth-ldap": {
"title": "LDAP auth"
},
"operations/basic-cluster-tuning": {
"title": "Basic cluster tuning"
},
@ -363,6 +369,12 @@
"operations/rule-configuration": {
"title": "Retaining or automatically dropping data"
},
"operations/security-overview": {
"title": "Security overview"
},
"operations/security-user-auth": {
"title": "User authentication and authorization"
},
"operations/segment-optimization": {
"title": "Segment Size Optimization"
},

35
website/sidebars.json
View File

@ -113,7 +113,28 @@
"Operations": [
"operations/druid-console",
"operations/getting-started",
"operations/basic-cluster-tuning",
{
"type": "subcategory",
"label": "Security",
"ids": [
"operations/security-overview",
"operations/security-user-auth",
"operations/auth-ldap",
"operations/password-provider",
"design/auth",
"operations/tls-support"
]
},
{
"type": "subcategory",
"label": "Performance tuning",
"ids": [
"operations/basic-cluster-tuning",
"operations/segment-optimization",
"operations/http-compression"
]
},
"operations/api-reference",
"operations/high-availability",
"operations/rolling-updates",
@ -121,22 +142,18 @@
"operations/metrics",
"operations/alerts",
"operations/other-hadoop",
"operations/http-compression",
"operations/tls-support",
"operations/password-provider",
"operations/dump-segment",
"operations/reset-cluster",
"operations/insert-segment-to-db",
"operations/pull-deps",
{
"type": "subcategory",
"label": "Misc",
"ids": [
"operations/management-uis",
"operations/dump-segment",
"operations/reset-cluster",
"operations/insert-segment-to-db",
"operations/pull-deps",
"operations/deep-storage-migration",
"operations/export-metadata",
"operations/metadata-migration",
"operations/segment-optimization",
"operations/use_sbt_to_build_fat_jar"
]
}