Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-9927. Document the new OIV ReverseXML processor (Wei-Chiu Chuang via cmccabe)

This commit is contained in:
Colin Patrick Mccabe 2016-03-10 13:38:00 -08:00
parent 017d2c127b
commit 500875dfcc
1 changed files with 10 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
View File

@ -38,7 +38,7 @@ The Offline Image Viewer provides several output processors:
interactively by using HTTP REST API. interactively by using HTTP REST API.
2. XML creates an XML document of the fsimage and includes all of the 2. XML creates an XML document of the fsimage and includes all of the
information within the fsimage, similar to the lsr processor. The information within the fsimage. The
output of this processor is amenable to automated processing and output of this processor is amenable to automated processing and
analysis with XML tools. Due to the verbosity of the XML syntax, analysis with XML tools. Due to the verbosity of the XML syntax,
this processor will also generate the largest amount of output. this processor will also generate the largest amount of output.
@ -60,6 +60,11 @@ The Offline Image Viewer provides several output processors:
delimiter. The default delimiter is \t, though this may be changed via delimiter. The default delimiter is \t, though this may be changed via
the -delimiter argument. the -delimiter argument.
5. ReverseXML (experimental): This is the opposite of the XML processor;
it reconstructs an fsimage from an XML file. This processor makes it easy to
create fsimages for testing, and manually edit fsimages when there is
corruption.
Usage Usage
----- -----
@ -139,9 +144,9 @@ Options
| **Flag** | **Description** | | **Flag** | **Description** |
|:---- |:---- | |:---- |:---- |
| `-i`\|`--inputFile` *input file* | Specify the input fsimage file to process. Required. | | `-i`\|`--inputFile` *input file* | Specify the input fsimage file (or XML file, if ReverseXML processor is used) to process. Required. |
| `-o`\|`--outputFile` *output file* | Specify the output filename, if the specified output processor generates one. If the specified file already exists, it is silently overwritten. (output to stdout by default)\| | `-o`\|`--outputFile` *output file* | Specify the output filename, if the specified output processor generates one. If the specified file already exists, it is silently overwritten. (output to stdout by default) If the input file is an XML file, it also creates an <outputFile>.md5. |
| `-p`\|`--processor` *processor* | Specify the image processor to apply against the image file. Currently valid options are Web (default), XML, Delimited and FileDistribution. | | `-p`\|`--processor` *processor* | Specify the image processor to apply against the image file. Currently valid options are `Web` (default), `XML`, `Delimited`, `FileDistribution` and `ReverseXML`. |
| `-addr` *address* | Specify the address(host:port) to listen. (localhost:5978 by default). This option is used with Web processor. | | `-addr` *address* | Specify the address(host:port) to listen. (localhost:5978 by default). This option is used with Web processor. |
| `-maxSize` *size* | Specify the range [0, maxSize] of file sizes to be analyzed in bytes (128GB by default). This option is used with FileDistribution processor. | | `-maxSize` *size* | Specify the range [0, maxSize] of file sizes to be analyzed in bytes (128GB by default). This option is used with FileDistribution processor. |
| `-step` *size* | Specify the granularity of the distribution in bytes (2MB by default). This option is used with FileDistribution processor. | | `-step` *size* | Specify the granularity of the distribution in bytes (2MB by default). This option is used with FileDistribution processor. |
@ -157,7 +162,7 @@ The Offline Image Viewer makes it easy to gather large amounts of data about the
oiv\_legacy Command oiv\_legacy Command
------------------- -------------------
Due to the internal layout changes introduced by the ProtocolBuffer-based fsimage ([HDFS-5698](https://issues.apache.org/jira/browse/HDFS-5698)), OfflineImageViewer consumes excessive amount of memory and loses some functions such as Indented and Delimited processor. If you want to process without large amount of memory or use these processors, you can use `oiv_legacy` command (same as `oiv` in Hadoop 2.3). Due to the internal layout changes introduced by the ProtocolBuffer-based fsimage ([HDFS-5698](https://issues.apache.org/jira/browse/HDFS-5698)), OfflineImageViewer consumes excessive amount of memory and loses some functions such as Indented processor. If you want to process without large amount of memory or use these processors, you can use `oiv_legacy` command (same as `oiv` in Hadoop 2.3).
### Usage ### Usage