diff --git a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt index 448f4348419..ae1cfa1f99f 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt +++ b/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt @@ -255,6 +255,9 @@ Release 2.5.0 - UNRELEASED IMPROVEMENTS + HDFS-6007. Update documentation about short-circuit local reads (iwasakims + via cmccabe) + OPTIMIZATIONS BUG FIXES diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml b/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml index b43e8b71196..fee8a58bb07 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml @@ -1416,17 +1416,6 @@ - - dfs.domain.socket.path - - - Optional. This is a path to a UNIX domain socket that will be used for - communication between the DataNode and local HDFS clients. - If the string "_PORT" is present in this path, it will be replaced by the - TCP port of the DataNode. - - - dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold 10737418240 @@ -1755,4 +1744,101 @@ + + dfs.client.read.shortcircuit + false + + This configuration parameter turns on short-circuit local reads. + + + + + dfs.domain.socket.path + + + Optional. This is a path to a UNIX domain socket that will be used for + communication between the DataNode and local HDFS clients. + If the string "_PORT" is present in this path, it will be replaced by the + TCP port of the DataNode. + + + + + dfs.client.read.shortcircuit.skip.checksum + false + + If this configuration parameter is set, + short-circuit local reads will skip checksums. + This is normally not recommended, + but it may be useful for special setups. + You might consider using this + if you are doing your own checksumming outside of HDFS. + + + + + dfs.client.read.shortcircuit.streams.cache.size + 256 + + The DFSClient maintains a cache of recently opened file descriptors. + This parameter controls the size of that cache. + Setting this higher will use more file descriptors, + but potentially provide better performance on workloads + involving lots of seeks. + + + + + dfs.client.read.shortcircuit.streams.cache.expiry.ms + 300000 + + This controls the minimum amount of time + file descriptors need to sit in the client cache context + before they can be closed for being inactive for too long. + + + + + dfs.datanode.shared.file.descriptor.paths + /dev/shm,/tmp + + Comma separated paths to the directory on which + shared memory segments are created. + The client and the DataNode exchange information via + this shared memory segment. + It tries paths in order until creation of shared memory segment succeeds. + + + + + dfs.client.use.legacy.blockreader.local + false + + Legacy short-circuit reader implementation based on HDFS-2246 is used + if this configuration parameter is true. + This is for the platforms other than Linux + where the new implementation based on HDFS-347 is not available. + + + + + dfs.block.local-path-access.user + + + Comma separated list of the users allowd to open block files + on legacy short-circuit local read. + + + + + dfs.client.domain.socket.data.traffic + false + + This control whether we will try to pass normal data traffic + over UNIX domain socket rather than over TCP socket + on node-local data transfer. + This is currently experimental and turned off by default. + + + diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ShortCircuitLocalReads.apt.vm b/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ShortCircuitLocalReads.apt.vm index 6670d8a1a65..8037203cbc9 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ShortCircuitLocalReads.apt.vm +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ShortCircuitLocalReads.apt.vm @@ -21,7 +21,9 @@ HDFS Short-Circuit Local Reads %{toc|section=1|fromDepth=0} -* {Background} +* {Short-Circuit Local Reads} + +** Background In <<>>, reads normally go through the <<>>. Thus, when the client asks the <<>> to read a file, the <<>> reads that @@ -31,7 +33,7 @@ HDFS Short-Circuit Local Reads the client is co-located with the data. Short-circuit reads provide a substantial performance boost to many applications. -* {Configuration} +** Setup To configure short-circuit local reads, you will need to enable <<>>. See @@ -39,16 +41,19 @@ HDFS Short-Circuit Local Reads Libraries}} for details on enabling this library. Short-circuit reads make use of a UNIX domain socket. This is a special path - in the filesystem that allows the client and the DataNodes to communicate. - You will need to set a path to this socket. The DataNode needs to be able to + in the filesystem that allows the client and the <<>>s to communicate. + You will need to set a path to this socket. The <<>> needs to be able to create this path. On the other hand, it should not be possible for any user - except the hdfs user or root to create this path. For this reason, paths + except the HDFS user or root to create this path. For this reason, paths under <<>> or <<>> are often used. + The client and the <<>> exchange information via a shared memory segment + on <<>>. + Short-circuit local reads need to be configured on both the <<>> and the client. -* {Example Configuration} +** Example Configuration Here is an example configuration. @@ -65,32 +70,43 @@ HDFS Short-Circuit Local Reads ---- -* {Configuration Keys} +* Legacy HDFS Short-Circuit Local Reads - * dfs.client.read.shortcircuit + Legacy implementation of short-circuit local reads + on which the clients directly open the HDFS block files + is still available for platforms other than the Linux. + Setting the value of <<>> + in addition to <<>> + to true enables this feature. - This configuration parameter turns on short-circuit local reads. + You also need to set the value of <<>> + to <<<750>>> instead of the default <<<700>>> and + chmod/chown the directory tree under <<>> + as readable to the client and the <<>>. + You must take caution because this means that + the client can read all of the block files bypassing HDFS permission. - * dfs.client.read.shortcircuit.skip.checksum + Because Legacy short-circuit local reads is insecure, + access to this feature is limited to the users listed in + the value of <<>>. - If this configuration parameter is set, short-circuit local reads will skip - checksums. This is normally not recommended, but it may be useful for - special setups. You might consider using this if you are doing your own - checksumming outside of HDFS. - - * dfs.client.read.shortcircuit.streams.cache.size - - The DFSClient maintains a cache of recently opened file descriptors. This - parameter controls the size of that cache. Setting this higher will use more - file descriptors, but potentially provide better performance on workloads - involving lots of seeks. - - * dfs.client.read.shortcircuit.streams.cache.expiry.ms - - This controls the minimum amount of time file descriptors need to sit in the - FileInputStreamCache before they can be closed for being inactive for too long. - - * dfs.client.domain.socket.data.traffic - - This control whether we will try to pass normal data traffic over UNIX domain - sockets. +---- + + + dfs.client.read.shortcircuit + true + + + dfs.client.use.legacy.blockreader.local + true + + + dfs.datanode.data.dir.perm + 750 + + + dfs.block.local-path-access.user + foo,bar + + +----