Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-9048. DistCp documentation is out-of-dated (Daisuke Kobayashi via iwasakims) 

(cherry picked from commit 33a412e8a4)
This commit is contained in:
Masatake Iwasaki 2016-03-03 18:57:23 +09:00
parent 9e87f59dea
commit 8095c612a3
2 changed files with 10 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -1963,6 +1963,9 @@ Release 2.7.3 - UNRELEASED
HDFS-8791. block ID-based DN storage layout can be very slow for datanode HDFS-8791. block ID-based DN storage layout can be very slow for datanode
on ext4 (Chris Trezzo via kihwal) on ext4 (Chris Trezzo via kihwal)
HDFS-9048. DistCp documentation is out-of-dated
(Daisuke Kobayashi via iwasakims)
OPTIMIZATIONS OPTIMIZATIONS
BUG FIXES BUG FIXES

13
hadoop-tools/hadoop-distcp/src/site/markdown/DistCp.md.vm
View File

@ -412,12 +412,13 @@ $H3 Map sizing
$H3 Copying Between Versions of HDFS $H3 Copying Between Versions of HDFS
For copying between two different versions of Hadoop, one will usually use For copying between two different major versions of Hadoop (e.g. between 1.X
HftpFileSystem. This is a read-only FileSystem, so DistCp must be run on the and 2.X), one will usually use WebHdfsFileSystem. Unlike the previous
destination cluster (more specifically, on NodeManagers that can write to the HftpFileSystem, as webhdfs is available for both read and write operations,
destination cluster). Each source is specified as DistCp can be run on both source and destination cluster.
`hftp://<dfs.http.address>/<path>` (the default `dfs.http.address` is Remote cluster is specified as `webhdfs://<namenode_hostname>:<http_port>`.
`<namenode>:50070`). When copying between same major versions of Hadoop cluster (e.g. between 2.X
and 2.X), use hdfs protocol for better performance.
$H3 MapReduce and other side-effects $H3 MapReduce and other side-effects