From c0756dfef61fb33ab940d45b37c4d49e95a86a7b Mon Sep 17 00:00:00 2001 From: Akira Ajisaka Date: Fri, 9 Sep 2016 01:34:34 +0900 Subject: [PATCH] HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin. (cherry picked from commit 20a20c2f6e1b3b8aa6a58a824ad3aadc349dc761) --- .../hadoop-hdfs/src/site/markdown/HDFSCommands.md | 3 +++ .../hadoop-hdfs/src/site/markdown/HdfsImageViewer.md | 12 +++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md index 6d5b8dbf600..b724871d3fb 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md @@ -251,6 +251,9 @@ Usage: `hdfs oiv_legacy [OPTIONS] -i INPUT_FILE -o OUTPUT_FILE` | COMMAND\_OPTION | Description | |:---- |:---- | | `-p`\|`--processor` *processor* | Specify the image processor to apply against the image file. Valid options are Ls (default), XML, Delimited, Indented, and FileDistribution. | +| `-maxSize` *size* | Specify the range [0, maxSize] of file sizes to be analyzed in bytes (128GB by default). This option is used with FileDistribution processor. | +| `-step` *size* | Specify the granularity of the distribution in bytes (2MB by default). This option is used with FileDistribution processor. | +| `-format` | Format the output result in a human-readable fashion rather than a number of bytes. (false by default). This option is used with FileDistribution processor. | | `-skipBlocks` | Do not enumerate individual blocks within files. This may save processing time and outfile file space on namespaces with very large files. The Ls processor reads the blocks to correctly determine file sizes and ignores this option. | | `-printToScreen` | Pipe output of processor to console as well as specified file. On extremely large namespaces, this may increase processing time by an order of magnitude. | | `-delimiter` *arg* | When used in conjunction with the Delimited processor, replaces the default tab delimiter with the string specified by *arg*. | diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md index f991b4495e7..f55c9fda024 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md @@ -50,10 +50,13 @@ The Offline Image Viewer provides several output processors: ..., s[n-1], maxSize], and the processor calculates how many files in the system fall into each segment [s[i-1], s[i]). Note that files larger than maxSize always fall into the very last segment. - The output file is formatted as a tab separated two column table: - Size and NumFiles. Where Size represents the start of the segment, + By default, the output file is formatted as a tab separated two column + table: Size and NumFiles. Where Size represents the start of the segment, and numFiles is the number of files form the image which size falls - in this segment. + in this segment. By specifying the option -format, the output file will be + formatted in a human-readable fashion rather than a number of bytes that + showed in Size column. In addition, the Size column will be changed to the + Size Range column. 4. Delimited (experimental): Generate a text file with all of the elements common to both inodes and inodes-under-construction, separated by a @@ -182,6 +185,9 @@ Due to the internal layout changes introduced by the ProtocolBuffer-based fsimag | `-i`\|`--inputFile` *input file* | Specify the input fsimage file to process. Required. | | `-o`\|`--outputFile` *output file* | Specify the output filename, if the specified output processor generates one. If the specified file already exists, it is silently overwritten. Required. | | `-p`\|`--processor` *processor* | Specify the image processor to apply against the image file. Valid options are Ls (default), XML, Delimited, Indented, and FileDistribution. | +| `-maxSize` *size* | Specify the range [0, maxSize] of file sizes to be analyzed in bytes (128GB by default). This option is used with FileDistribution processor. | +| `-step` *size* | Specify the granularity of the distribution in bytes (2MB by default). This option is used with FileDistribution processor. | +| `-format` | Format the output result in a human-readable fashion rather than a number of bytes. (false by default). This option is used with FileDistribution processor. | | `-skipBlocks` | Do not enumerate individual blocks within files. This may save processing time and outfile file space on namespaces with very large files. The Ls processor reads the blocks to correctly determine file sizes and ignores this option. | | `-printToScreen` | Pipe output of processor to console as well as specified file. On extremely large namespaces, this may increase processing time by an order of magnitude. | | `-delimiter` *arg* | When used in conjunction with the Delimited processor, replaces the default tab delimiter with the string specified by *arg*. |