HDFS-7195. Update user doc of secure mode about Datanodes don't require root or jsvc. Contributed by Chris Nauroth.
(cherry picked from commit 4e544c84e9)
This commit is contained in:
cnauroth
parent
cf0ad82c5a
commit
754718aed5
|
|
@ -24,7 +24,10 @@
|
|||
# The java implementation to use.
|
||||
export JAVA_HOME=${JAVA_HOME}
|
||||
|
||||
# The jsvc implementation to use. Jsvc is required to run secure datanodes.
|
||||
# The jsvc implementation to use. Jsvc is required to run secure datanodes
|
||||
# that bind to privileged ports to provide authentication of data transfer
|
||||
# protocol. Jsvc is not required if SASL is configured for authentication of
|
||||
# data transfer protocol using non-privileged ports.
|
||||
#export JSVC_HOME=${JSVC_HOME}
|
||||
|
||||
export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/etc/hadoop"}
|
||||
|
|
@ -58,7 +61,11 @@ export HADOOP_PORTMAP_OPTS="-Xmx512m $HADOOP_PORTMAP_OPTS"
|
|||
export HADOOP_CLIENT_OPTS="-Xmx512m $HADOOP_CLIENT_OPTS"
|
||||
#HADOOP_JAVA_PLATFORM_OPTS="-XX:-UsePerfData $HADOOP_JAVA_PLATFORM_OPTS"
|
||||
|
||||
# On secure datanodes, user to run the datanode as after dropping privileges
|
||||
# On secure datanodes, user to run the datanode as after dropping privileges.
|
||||
# This **MUST** be uncommented to enable secure HDFS if using privileged ports
|
||||
# to provide authentication of data transfer protocol. This **MUST NOT** be
|
||||
# defined if SASL is configured for authentication of data transfer protocol
|
||||
# using non-privileged ports.
|
||||
export HADOOP_SECURE_DN_USER=${HADOOP_SECURE_DN_USER}
|
||||
|
||||
# Where log files are stored. $HADOOP_HOME/logs by default.
|
||||
|
|
|
|||
|
|
@ -244,7 +244,7 @@ KVNO Timestamp Principal
|
|||
10.222.0.0-15 and 10.113.221.221
|
||||
can impersonate any user belonging to any group.
|
||||
|
||||
----
|
||||
----
|
||||
<property>
|
||||
<name>hadoop.proxyuser.oozie.hosts</name>
|
||||
<value>10.222.0.0/16,10.113.221.221</value>
|
||||
|
|
@ -273,6 +273,33 @@ KVNO Timestamp Principal
|
|||
You must specify <<<HADOOP_SECURE_DN_USER>>> and <<<JSVC_HOME>>>
|
||||
as environment variables on start up (in hadoop-env.sh).
|
||||
|
||||
As of version 2.6.0, SASL can be used to authenticate the data transfer
|
||||
protocol. In this configuration, it is no longer required for secured clusters
|
||||
to start the DataNode as root using jsvc and bind to privileged ports. To
|
||||
enable SASL on data transfer protocol, set <<<dfs.data.transfer.protection>>>
|
||||
in hdfs-site.xml, set a non-privileged port for <<<dfs.datanode.address>>>, set
|
||||
<<<dfs.http.policy>>> to <HTTPS_ONLY> and make sure the
|
||||
<<<HADOOP_SECURE_DN_USER>>> environment variable is not defined. Note that it
|
||||
is not possible to use SASL on data transfer protocol if
|
||||
<<<dfs.datanode.address>>> is set to a privileged port. This is required for
|
||||
backwards-compatibility reasons.
|
||||
|
||||
In order to migrate an existing cluster that used root authentication to start
|
||||
using SASL instead, first ensure that version 2.6.0 or later has been deployed
|
||||
to all cluster nodes as well as any external applications that need to connect
|
||||
to the cluster. Only versions 2.6.0 and later of the HDFS client can connect
|
||||
to a DataNode that uses SASL for authentication of data transfer protocol, so
|
||||
it is vital that all callers have the correct version before migrating. After
|
||||
version 2.6.0 or later has been deployed everywhere, update configuration of
|
||||
any external applications to enable SASL. If an HDFS client is enabled for
|
||||
SASL, then it can connect successfully to a DataNode running with either root
|
||||
authentication or SASL authentication. Changing configuration for all clients
|
||||
guarantees that subsequent configuration changes on DataNodes will not disrupt
|
||||
the applications. Finally, each individual DataNode can be migrated by
|
||||
changing its configuration and restarting. It is acceptable to have a mix of
|
||||
some DataNodes running with root authentication and some DataNodes running with
|
||||
SASL authentication temporarily during this migration period, because an HDFS
|
||||
client enabled for SASL can connect to both.
|
||||
|
||||
* Data confidentiality
|
||||
|
||||
|
|
@ -391,6 +418,10 @@ Configuration for <<<conf/core-site.xml>>>
|
|||
| <<<dfs.http.policy>>> | <HTTP_ONLY> or <HTTPS_ONLY> or <HTTP_AND_HTTPS> | |
|
||||
| | | HTTPS_ONLY turns off http access. This option takes precedence over |
|
||||
| | | the deprecated configuration dfs.https.enable and hadoop.ssl.enabled. |
|
||||
| | | If using SASL to authenticate data transfer protocol instead of |
|
||||
| | | running DataNode as root and using privileged ports, then this property |
|
||||
| | | must be set to <HTTPS_ONLY> to guarantee authentication of HTTP servers. |
|
||||
| | | (See <<<dfs.data.transfer.protection>>>.) |
|
||||
*-------------------------+-------------------------+------------------------+
|
||||
| <<<dfs.namenode.https-address>>> | <nn_host_fqdn:50470> | |
|
||||
*-------------------------+-------------------------+------------------------+
|
||||
|
|
@ -440,6 +471,9 @@ Configuration for <<<conf/hdfs-site.xml>>>
|
|||
| | | Secure DataNode must use privileged port |
|
||||
| | | in order to assure that the server was started securely. |
|
||||
| | | This means that the server must be started via jsvc. |
|
||||
| | | Alternatively, this must be set to a non-privileged port if using SASL |
|
||||
| | | to authenticate data transfer protocol. |
|
||||
| | | (See <<<dfs.data.transfer.protection>>>.) |
|
||||
*-------------------------+-------------------------+------------------------+
|
||||
| <<<dfs.datanode.http.address>>> | <0.0.0.0:1006> | |
|
||||
| | | Secure DataNode must use privileged port |
|
||||
|
|
@ -457,6 +491,17 @@ Configuration for <<<conf/hdfs-site.xml>>>
|
|||
| <<<dfs.encrypt.data.transfer>>> | <false> | |
|
||||
| | | set to <<<true>>> when using data encryption |
|
||||
*-------------------------+-------------------------+------------------------+
|
||||
| <<<dfs.data.transfer.protection>>> | | |
|
||||
| | | <authentication> : authentication only \
|
||||
| | | <integrity> : integrity check in addition to authentication \
|
||||
| | | <privacy> : data encryption in addition to integrity |
|
||||
| | | This property is unspecified by default. Setting this property enables |
|
||||
| | | SASL for authentication of data transfer protocol. If this is enabled, |
|
||||
| | | then <<<dfs.datanode.address>>> must use a non-privileged port, |
|
||||
| | | <<<dfs.http.policy>>> must be set to <HTTPS_ONLY> and the |
|
||||
| | | <<<HADOOP_SECURE_DN_USER>>> environment variable must be undefined when |
|
||||
| | | starting the DataNode process. |
|
||||
*-------------------------+-------------------------+------------------------+
|
||||
Configuration for <<<conf/hdfs-site.xml>>>
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -280,6 +280,9 @@ Release 2.6.0 - UNRELEASED
|
|||
|
||||
HDFS-7217. Better batching of IBRs. (kihwal)
|
||||
|
||||
HDFS-7195. Update user doc of secure mode about Datanodes don't require root
|
||||
or jsvc. (cnauroth)
|
||||
|
||||
OPTIMIZATIONS
|
||||
|
||||
HDFS-6690. Deduplicate xattr names in memory. (wang)
|
||||
|
|
|
|||
Loading…
Reference in New Issue