Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HDFS-9884. Use doxia macro to generate in-page TOC of HDFS site documentation. (iwasakims)

This commit is contained in:
Masatake Iwasaki 2017-01-28 08:11:40 +09:00
parent d20dc8691e
commit 528bff9c42
25 changed files with 26 additions and 482 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md
View File

@ -15,19 +15,7 @@
Archival Storage, SSD & Memory
==============================
* [Archival Storage, SSD & Memory](#Archival_Storage_SSD__Memory)
* [Introduction](#Introduction)
* [Storage Types and Storage Policies](#Storage_Types_and_Storage_Policies)
* [Storage Types: ARCHIVE, DISK, SSD and RAM\_DISK](#Storage_Types:_ARCHIVE_DISK_SSD_and_RAM_DISK)
* [Storage Policies: Hot, Warm, Cold, All\_SSD, One\_SSD and Lazy\_Persist](#Storage_Policies:_Hot_Warm_Cold_All_SSD_One_SSD_and_Lazy_Persist)
* [Storage Policy Resolution](#Storage_Policy_Resolution)
* [Configuration](#Configuration)
* [Mover - A New Data Migration Tool](#Mover_-_A_New_Data_Migration_Tool)
* [Storage Policy Commands](#Storage_Policy_Commands)
* [List Storage Policies](#List_Storage_Policies)
* [Set Storage Policy](#Set_Storage_Policy)
* [Unset Storage Policy](#Unset_Storage_Policy)
* [Get Storage Policy](#Get_Storage_Policy)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Introduction
------------

25
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md
View File

@ -15,30 +15,7 @@
Centralized Cache Management in HDFS
====================================
* [Overview](#Overview)
* [Use Cases](#Use_Cases)
* [Architecture](#Architecture)
* [Concepts](#Concepts)
* [Cache directive](#Cache_directive)
* [Cache pool](#Cache_pool)
* [cacheadmin command-line interface](#cacheadmin_command-line_interface)
* [Cache directive commands](#Cache_directive_commands)
* [addDirective](#addDirective)
* [removeDirective](#removeDirective)
* [removeDirectives](#removeDirectives)
* [listDirectives](#listDirectives)
* [Cache pool commands](#Cache_pool_commands)
* [addPool](#addPool)
* [modifyPool](#modifyPool)
* [removePool](#removePool)
* [listPools](#listPools)
* [help](#help)
* [Configuration](#Configuration)
* [Native Libraries](#Native_Libraries)
* [Configuration Properties](#Configuration_Properties)
* [Required](#Required)
* [Optional](#Optional)
* [OS Limits](#OS_Limits)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

8
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ExtendedAttributes.md
View File

@ -15,13 +15,7 @@
Extended Attributes in HDFS
===========================
* [Overview](#Overview)
* [HDFS extended attributes](#HDFS_extended_attributes)
* [Namespaces and Permissions](#Namespaces_and_Permissions)
* [Interacting with extended attributes](#Interacting_with_extended_attributes)
* [getfattr](#getfattr)
* [setfattr](#setfattr)
* [Configuration options](#Configuration_options)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

15
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Federation.md
View File

@ -15,20 +15,7 @@
HDFS Federation
===============
* [HDFS Federation](#HDFS_Federation)
* [Background](#Background)
* [Multiple Namenodes/Namespaces](#Multiple_NamenodesNamespaces)
* [Key Benefits](#Key_Benefits)
* [Federation Configuration](#Federation_Configuration)
* [Configuration:](#Configuration:)
* [Formatting Namenodes](#Formatting_Namenodes)
* [Upgrading from an older release and configuring federation](#Upgrading_from_an_older_release_and_configuring_federation)
* [Adding a new Namenode to an existing HDFS cluster](#Adding_a_new_Namenode_to_an_existing_HDFS_cluster)
* [Managing the cluster](#Managing_the_cluster)
* [Starting and stopping cluster](#Starting_and_stopping_cluster)
* [Balancer](#Balancer)
* [Decommissioning](#Decommissioning)
* [Cluster Web Console](#Cluster_Web_Console)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
This guide provides an overview of the HDFS Federation feature and how to configure and manage the federated cluster.

35
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
View File

@ -15,40 +15,7 @@
HDFS Commands Guide
===================
* [Overview](#Overview)
* [User Commands](#User_Commands)
* [classpath](#classpath)
* [dfs](#dfs)
* [fetchdt](#fetchdt)
* [fsck](#fsck)
* [getconf](#getconf)
* [groups](#groups)
* [lsSnapshottableDir](#lsSnapshottableDir)
* [jmxget](#jmxget)
* [oev](#oev)
* [oiv](#oiv)
* [oiv\_legacy](#oiv_legacy)
* [snapshotDiff](#snapshotDiff)
* [version](#version)
* [Administration Commands](#Administration_Commands)
* [balancer](#balancer)
* [cacheadmin](#cacheadmin)
* [crypto](#crypto)
* [datanode](#datanode)
* [dfsadmin](#dfsadmin)
* [haadmin](#haadmin)
* [journalnode](#journalnode)
* [mover](#mover)
* [namenode](#namenode)
* [nfs3](#nfs3)
* [portmap](#portmap)
* [secondarynamenode](#secondarynamenode)
* [storagepolicies](#storagepolicies)
* [zkfc](#zkfc)
* [Debug Commands](#Debug_Commands)
* [verifyMeta](#verifyMeta)
* [computeMeta](#computeMeta)
* [recoverLease](#recoverLease)
<!-- MACRO{toc|fromDepth=0|toDepth=2} -->
Overview
--------

25
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md
View File

@ -15,30 +15,7 @@
HDFS High Availability
======================
* [HDFS High Availability](#HDFS_High_Availability)
* [Purpose](#Purpose)
* [Note: Using the Quorum Journal Manager or Conventional Shared Storage](#Note:_Using_the_Quorum_Journal_Manager_or_Conventional_Shared_Storage)
* [Background](#Background)
* [Architecture](#Architecture)
* [Hardware resources](#Hardware_resources)
* [Deployment](#Deployment)
* [Configuration overview](#Configuration_overview)
* [Configuration details](#Configuration_details)
* [Deployment details](#Deployment_details)
* [Administrative commands](#Administrative_commands)
* [Automatic Failover](#Automatic_Failover)
* [Introduction](#Introduction)
* [Components](#Components)
* [Deploying ZooKeeper](#Deploying_ZooKeeper)
* [Before you begin](#Before_you_begin)
* [Configuring automatic failover](#Configuring_automatic_failover)
* [Initializing HA state in ZooKeeper](#Initializing_HA_state_in_ZooKeeper)
* [Starting the cluster with start-dfs.sh](#Starting_the_cluster_with_start-dfs.sh)
* [Starting the cluster manually](#Starting_the_cluster_manually)
* [Securing access to ZooKeeper](#Securing_access_to_ZooKeeper)
* [Verifying automatic failover](#Verifying_automatic_failover)
* [Automatic Failover FAQ](#Automatic_Failover_FAQ)
* [BookKeeper as a Shared storage (EXPERIMENTAL)](#BookKeeper_as_a_Shared_storage_EXPERIMENTAL)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Purpose
-------

25
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md
View File

@ -15,30 +15,7 @@
HDFS High Availability Using the Quorum Journal Manager
=======================================================
* [HDFS High Availability Using the Quorum Journal Manager](#HDFS_High_Availability_Using_the_Quorum_Journal_Manager)
* [Purpose](#Purpose)
* [Note: Using the Quorum Journal Manager or Conventional Shared Storage](#Note:_Using_the_Quorum_Journal_Manager_or_Conventional_Shared_Storage)
* [Background](#Background)
* [Architecture](#Architecture)
* [Hardware resources](#Hardware_resources)
* [Deployment](#Deployment)
* [Configuration overview](#Configuration_overview)
* [Configuration details](#Configuration_details)
* [Deployment details](#Deployment_details)
* [Administrative commands](#Administrative_commands)
* [Automatic Failover](#Automatic_Failover)
* [Introduction](#Introduction)
* [Components](#Components)
* [Deploying ZooKeeper](#Deploying_ZooKeeper)
* [Before you begin](#Before_you_begin)
* [Configuring automatic failover](#Configuring_automatic_failover)
* [Initializing HA state in ZooKeeper](#Initializing_HA_state_in_ZooKeeper)
* [Starting the cluster with start-dfs.sh](#Starting_the_cluster_with_start-dfs.sh)
* [Starting the cluster manually](#Starting_the_cluster_manually)
* [Securing access to ZooKeeper](#Securing_access_to_ZooKeeper)
* [Verifying automatic failover](#Verifying_automatic_failover)
* [Automatic Failover FAQ](#Automatic_Failover_FAQ)
* [HDFS Upgrade/Finalization/Rollback with HA Enabled](#HDFS_UpgradeFinalizationRollback_with_HA_Enabled)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Purpose
-------

36
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
View File

@ -15,41 +15,7 @@
HDFS Architecture
=================
* [HDFS Architecture](#HDFS_Architecture)
* [Introduction](#Introduction)
* [Assumptions and Goals](#Assumptions_and_Goals)
* [Hardware Failure](#Hardware_Failure)
* [Streaming Data Access](#Streaming_Data_Access)
* [Large Data Sets](#Large_Data_Sets)
* [Simple Coherency Model](#Simple_Coherency_Model)
* ["Moving Computation is Cheaper than Moving Data"](#aMoving_Computation_is_Cheaper_than_Moving_Data)
* [Portability Across Heterogeneous Hardware and Software Platforms](#Portability_Across_Heterogeneous_Hardware_and_Software_Platforms)
* [NameNode and DataNodes](#NameNode_and_DataNodes)
* [The File System Namespace](#The_File_System_Namespace)
* [Data Replication](#Data_Replication)
* [Replica Placement: The First Baby Steps](#Replica_Placement:_The_First_Baby_Steps)
* [Replica Selection](#Replica_Selection)
* [Safemode](#Safemode)
* [The Persistence of File System Metadata](#The_Persistence_of_File_System_Metadata)
* [The Communication Protocols](#The_Communication_Protocols)
* [Robustness](#Robustness)
* [Data Disk Failure, Heartbeats and Re-Replication](#Data_Disk_Failure_Heartbeats_and_Re-Replication)
* [Cluster Rebalancing](#Cluster_Rebalancing)
* [Data Integrity](#Data_Integrity)
* [Metadata Disk Failure](#Metadata_Disk_Failure)
* [Snapshots](#Snapshots)
* [Data Organization](#Data_Organization)
* [Data Blocks](#Data_Blocks)
* [Staging](#Staging)
* [Replication Pipelining](#Replication_Pipelining)
* [Accessibility](#Accessibility)
* [FS Shell](#FS_Shell)
* [DFSAdmin](#DFSAdmin)
* [Browser Interface](#Browser_Interface)
* [Space Reclamation](#Space_Reclamation)
* [File Deletes and Undeletes](#File_Deletes_and_Undeletes)
* [Decrease Replication Factor](#Decrease_Replication_Factor)
* [References](#References)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Introduction
------------

5
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md
View File

@ -15,10 +15,7 @@
Offline Edits Viewer Guide
==========================
* [Offline Edits Viewer Guide](#Offline_Edits_Viewer_Guide)
* [Overview](#Overview)
* [Usage](#Usage)
* [Case study: Hadoop cluster recovery](#Case_study:_Hadoop_cluster_recovery)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

11
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md
View File

@ -15,16 +15,7 @@
Offline Image Viewer Guide
==========================
* [Offline Image Viewer Guide](#Offline_Image_Viewer_Guide)
* [Overview](#Overview)
* [Usage](#Usage)
* [Web Processor](#Web_Processor)
* [XML Processor](#XML_Processor)
* [Options](#Options)
* [Analyzing Results](#Analyzing_Results)
* [oiv\_legacy Command](#oiv_legacy_Command)
* [Usage](#Usage)
* [Options](#Options)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

9
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md
View File

@ -17,14 +17,7 @@ HDFS Support for Multihomed Networks
This document is targetted to cluster administrators deploying `HDFS` in multihomed networks. Similar support for `YARN`/`MapReduce` is work in progress and will be documented when available.
* [HDFS Support for Multihomed Networks](#HDFS_Support_for_Multihomed_Networks)
* [Multihoming Background](#Multihoming_Background)
* [Fixing Hadoop Issues In Multihomed Environments](#Fixing_Hadoop_Issues_In_Multihomed_Environments)
* [Ensuring HDFS Daemons Bind All Interfaces](#Ensuring_HDFS_Daemons_Bind_All_Interfaces)
* [Clients use Hostnames when connecting to DataNodes](#Clients_use_Hostnames_when_connecting_to_DataNodes)
* [DataNodes use HostNames when connecting to other DataNodes](#DataNodes_use_HostNames_when_connecting_to_other_DataNodes)
* [Multihoming and Hadoop Security](#Multihoming_and_Hadoop_Security)
* [Hostname Lookup](#Hostname_Lookup)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Multihoming Background
----------------------

9
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md
View File

@ -15,14 +15,7 @@
HDFS NFS Gateway
================
* [HDFS NFS Gateway](#HDFS_NFS_Gateway)
* [Overview](#Overview)
* [Configuration](#Configuration)
* [Start and stop NFS gateway service](#Start_and_stop_NFS_gateway_service)
* [Verify validity of NFS related services](#Verify_validity_of_NFS_related_services)
* [Mount the export "/"](#Mount_the_export_)
* [Allow mounts from unprivileged clients](#Allow_mounts_from_unprivileged_clients)
* [User authentication and mapping](#User_authentication_and_mapping)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

15
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsPermissionsGuide.md
View File

@ -15,20 +15,7 @@
HDFS Permissions Guide
======================
* [HDFS Permissions Guide](#HDFS_Permissions_Guide)
* [Overview](#Overview)
* [User Identity](#User_Identity)
* [Group Mapping](#Group_Mapping)
* [Permission Checks](#Permission_Checks)
* [Understanding the Implementation](#Understanding_the_Implementation)
* [Changes to the File System API](#Changes_to_the_File_System_API)
* [Changes to the Application Shell](#Changes_to_the_Application_Shell)
* [The Super-User](#The_Super-User)
* [The Web Server](#The_Web_Server)
* [ACLs (Access Control Lists)](#ACLs_Access_Control_Lists)
* [ACLs File System API](#ACLs_File_System_API)
* [ACLs Shell Commands](#ACLs_Shell_Commands)
* [Configuration Parameters](#Configuration_Parameters)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

10
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsQuotaAdminGuide.md
View File

@ -15,13 +15,7 @@
HDFS Quotas Guide
=================
* [HDFS Quotas Guide](#HDFS_Quotas_Guide)
* [Overview](#Overview)
* [Name Quotas](#Name_Quotas)
* [Space Quotas](#Space_Quotas)
* [Storage Type Quotas](#Storage_Type_Quotas)
* [Administrative Commands](#Administrative_Commands)
* [Reporting Command](#Reporting_Command)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------
@ -128,4 +122,4 @@ An an extension to the count command of the HDFS shell reports quota values and
storage type quota set and the available quota remaining for each directory.
If specific storage types are given after -t option, only quota and remaining
quota of the types specified will be displayed. Otherwise, quota and
remaining quota of all storage types that support quota will be displayed.
remaining quota of all storage types that support quota will be displayed.

20
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsRollingUpgrade.md
View File

@ -18,25 +18,7 @@
HDFS Rolling Upgrade
====================
* [Introduction](#Introduction)
* [Upgrade](#Upgrade)
* [Upgrade without Downtime](#Upgrade_without_Downtime)
* [Upgrading Non-Federated Clusters](#Upgrading_Non-Federated_Clusters)
* [Upgrading Federated Clusters](#Upgrading_Federated_Clusters)
* [Upgrade with Downtime](#Upgrade_with_Downtime)
* [Upgrading Non-HA Clusters](#Upgrading_Non-HA_Clusters)
* [Downgrade and Rollback](#Downgrade_and_Rollback)
* [Downgrade](#Downgrade)
* [Downgrade without Downtime](#Downgrade_without_Downtime)
* [Downgrade with Downtime](#Downgrade_with_Downtime)
* [Rollback](#Rollback)
* [Commands and Startup Options for Rolling Upgrade](#Commands_and_Startup_Options_for_Rolling_Upgrade)
* [DFSAdmin Commands](#DFSAdmin_Commands)
* [dfsadmin -rollingUpgrade](#dfsadmin_-rollingUpgrade)
* [dfsadmin -getDatanodeInfo](#dfsadmin_-getDatanodeInfo)
* [dfsadmin -shutdownDatanode](#dfsadmin_-shutdownDatanode)
* [NameNode Startup Options](#NameNode_Startup_Options)
* [namenode -rollingUpgrade](#namenode_-rollingUpgrade)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Introduction

16
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsSnapshots.md
View File

@ -18,21 +18,7 @@
HDFS Snapshots
==============
* [HDFS Snapshots](#HDFS_Snapshots)
* [Overview](#Overview)
* [Snapshottable Directories](#Snapshottable_Directories)
* [Snapshot Paths](#Snapshot_Paths)
* [Upgrading to a version of HDFS with snapshots](#Upgrading_to_a_version_of_HDFS_with_snapshots)
* [Snapshot Operations](#Snapshot_Operations)
* [Administrator Operations](#Administrator_Operations)
* [Allow Snapshots](#Allow_Snapshots)
* [Disallow Snapshots](#Disallow_Snapshots)
* [User Operations](#User_Operations)
* [Create Snapshots](#Create_Snapshots)
* [Delete Snapshots](#Delete_Snapshots)
* [Rename Snapshots](#Rename_Snapshots)
* [Get Snapshottable Directory Listing](#Get_Snapshottable_Directory_Listing)
* [Get Snapshots Difference Report](#Get_Snapshots_Difference_Report)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview

23
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md
View File

@ -15,28 +15,7 @@
HDFS Users Guide
================
* [HDFS Users Guide](#HDFS_Users_Guide)
* [Purpose](#Purpose)
* [Overview](#Overview)
* [Prerequisites](#Prerequisites)
* [Web Interface](#Web_Interface)
* [Shell Commands](#Shell_Commands)
* [DFSAdmin Command](#DFSAdmin_Command)
* [Secondary NameNode](#Secondary_NameNode)
* [Checkpoint Node](#Checkpoint_Node)
* [Backup Node](#Backup_Node)
* [Import Checkpoint](#Import_Checkpoint)
* [Balancer](#Balancer)
* [Rack Awareness](#Rack_Awareness)
* [Safemode](#Safemode)
* [fsck](#fsck)
* [fetchdt](#fetchdt)
* [Recovery Mode](#Recovery_Mode)
* [Upgrade and Rollback](#Upgrade_and_Rollback)
* [DataNode Hot Swap Drive](#DataNode_Hot_Swap_Drive)
* [File Permissions and Security](#File_Permissions_and_Security)
* [Scalability](#Scalability)
* [Related Documentation](#Related_Documentation)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Purpose
-------

6
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Hftp.md
View File

@ -15,11 +15,7 @@
HFTP Guide
==========
* [HFTP Guide](#HFTP_Guilde)
* [Deprecated](#Deprecated)
* [Introduction](#Introduction)
* [Implementation](#Implementation)
* [Configuration Options](#Configuration_Options)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Deprecated

8
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/LibHdfs.md
View File

@ -15,13 +15,7 @@
C API libhdfs
=============
* [C API libhdfs](#C_API_libhdfs)
* [Overview](#Overview)
* [The APIs](#The_APIs)
* [A Sample Program](#A_Sample_Program)
* [How To Link With The Library](#How_To_Link_With_The_Library)
* [Common Problems](#Common_Problems)
* [Thread Safe](#Thread_Safe)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

14
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/MemoryStorage.md
View File

@ -15,19 +15,7 @@
Memory Storage Support in HDFS
==============================
* [Introduction](#Introduction)
* [Administrator Configuration](#Administrator_Configuration)
* [Limit RAM used for replicas in Memory](#Limit_RAM_used_for_replicas_in_Memory)
* [Setup RAM Disks on Data Nodes](#Setup_RAM_Disks_on_Data_Nodes)
* [Choosing `tmpfs` \(vs `ramfs`\)](#Choosing_`tmpfs`_\(vs_`ramfs`\))
* [Mount RAM Disks](#Mount_RAM_Disks)
* [Tag `tmpfs` volume with the RAM\_DISK Storage Type](#Tag_`tmpfs`_volume_with_the_RAM\_DISK_Storage_Type)
* [Ensure Storage Policies are enabled](#Ensure_Storage_Policies_are_enabled)
* [Application Usage](#Application_Usage)
* [Use the LAZY\_PERSIST Storage Policy](#Use_the_LAZY\_PERSIST_Storage_Policy)
* [Invoke `hdfs storagepolicies` command for directories](#Invoke_hdfs_storagepolicies_command_for_directories)
* [Call `setStoragePolicy` method for directories](#Call_`setStoragePolicy`_method_for_directories)
* [Pass `LAZY_PERSIST` `CreateFlag` for new files](#Pass_`LAZY_PERSIST`_`CreateFlag`_for_new_files)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Introduction
------------

7
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/SLGUserGuide.md
View File

@ -15,12 +15,7 @@
Synthetic Load Generator Guide
==============================
* [Synthetic Load Generator Guide](#Synthetic_Load_Generator_Guide)
* [Overview](#Overview)
* [Synopsis](#Synopsis)
* [Test Space Population](#Test_Space_Population)
* [Structure Generator](#Structure_Generator)
* [Data Generator](#Data_Generator)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Overview
--------

7
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ShortCircuitLocalReads.md
View File

@ -15,12 +15,7 @@
HDFS Short-Circuit Local Reads
==============================
* [HDFS Short-Circuit Local Reads](#HDFS_Short-Circuit_Local_Reads)
* [Short-Circuit Local Reads](#Short-Circuit_Local_Reads)
* [Background](#Background)
* [Setup](#Setup)
* [Example Configuration](#Example_Configuration)
* [Legacy HDFS Short-Circuit Local Reads](#Legacy_HDFS_Short-Circuit_Local_Reads)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Short-Circuit Local Reads
-------------------------

27
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md
View File

@ -15,32 +15,7 @@
Transparent Encryption in HDFS
==============================
* [Overview](#Overview)
* [Background](#Background)
* [Use Cases](#Use_Cases)
* [Architecture](#Architecture)
* [Overview](#Architecture_overview)
* [Accessing data within an encryption zone](#Accessing_data_within_an_encryption_zone)
* [Key Management Server, KeyProvider, EDEKs](#Key_Management_Server_KeyProvider_EDEKs)
* [Configuration](#Configuration)
* [Configuring the cluster KeyProvider](#Configuring_the_cluster_KeyProvider)
* [Selecting an encryption algorithm and codec](#Selecting_an_encryption_algorithm_and_codec)
* [Namenode configuration](#Namenode_configuration)
* [crypto command-line interface](#crypto_command-line_interface)
* [createZone](#createZone)
* [listZones](#listZones)
* [provisionTrash](#provisionTrash)
* [getFileEncryptionInfo](#getFileEncryptionInfo)
* [Example usage](#Example_usage)
* [Distcp considerations](#Distcp_considerations)
* [Running as the superuser](#Running_as_the_superuser)
* [Copying into encrypted locations](#Copying_into_encrypted_locations)
* [Rename and Trash considerations](#Rename_and_Trash_considerations)
* [Attack vectors](#Attack_vectors)
* [Hardware access exploits](#Hardware_access_exploits)
* [Root access exploits](#Root_access_exploits)
* [HDFS admin exploits](#HDFS_admin_exploits)
* [Rogue user exploits](#Rogue_user_exploits)
<!-- MACRO{toc|fromDepth=0|toDepth=2} -->
<a name="Overview"></a>Overview
--------

15
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md
View File

@ -15,20 +15,7 @@
ViewFs Guide
============
* [ViewFs Guide](#ViewFs_Guide)
* [Introduction](#Introduction)
* [The Old World (Prior to Federation)](#The_Old_World_Prior_to_Federation)
* [Single Namenode Clusters](#Single_Namenode_Clusters)
* [Pathnames Usage Patterns](#Pathnames_Usage_Patterns)
* [Pathname Usage Best Practices](#Pathname_Usage_Best_Practices)
* [New World – Federation and ViewFs](#New_World__Federation_and_ViewFs)
* [How The Clusters Look](#How_The_Clusters_Look)
* [A Global Namespace Per Cluster Using ViewFs](#A_Global_Namespace_Per_Cluster_Using_ViewFs)
* [Pathname Usage Patterns](#Pathname_Usage_Patterns)
* [Pathname Usage Best Practices](#Pathname_Usage_Best_Practices)
* [Renaming Pathnames Across Namespaces](#Renaming_Pathnames_Across_Namespaces)
* [FAQ](#FAQ)
* [Appendix: A Mount Table Configuration Example](#Appendix:_A_Mount_Table_Configuration_Example)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Introduction
------------

123
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md
View File

@ -15,128 +15,7 @@
WebHDFS REST API
================
* [WebHDFS REST API](#WebHDFS_REST_API)
* [Document Conventions](#Document_Conventions)
* [Introduction](#Introduction)
* [Operations](#Operations)
* [FileSystem URIs vs HTTP URLs](#FileSystem_URIs_vs_HTTP_URLs)
* [HDFS Configuration Options](#HDFS_Configuration_Options)
* [Authentication](#Authentication)
* [Proxy Users](#Proxy_Users)
* [Cross-Site Request Forgery Prevention](#Cross-Site_Request_Forgery_Prevention)
* [WebHDFS Retry Policy](#WebHDFS_Retry_Policy)
* [File and Directory Operations](#File_and_Directory_Operations)
* [Create and Write to a File](#Create_and_Write_to_a_File)
* [Append to a File](#Append_to_a_File)
* [Concat File(s)](#Concat_Files)
* [Open and Read a File](#Open_and_Read_a_File)
* [Make a Directory](#Make_a_Directory)
* [Create a Symbolic Link](#Create_a_Symbolic_Link)
* [Rename a File/Directory](#Rename_a_FileDirectory)
* [Delete a File/Directory](#Delete_a_FileDirectory)
* [Truncate a File](#Truncate_a_File)
* [Status of a File/Directory](#Status_of_a_FileDirectory)
* [List a Directory](#List_a_Directory)
* [Iteratively List a Directory](#Iteratively_List_a_Directory)
* [Get File Block Locations](#Get_File_Block_Locations)
* [Other File System Operations](#Other_File_System_Operations)
* [Get Content Summary of a Directory](#Get_Content_Summary_of_a_Directory)
* [Get File Checksum](#Get_File_Checksum)
* [Get Home Directory](#Get_Home_Directory)
* [Get Trash Root](#Get_Trash_Root)
* [Set Permission](#Set_Permission)
* [Set Owner](#Set_Owner)
* [Set Replication Factor](#Set_Replication_Factor)
* [Set Access or Modification Time](#Set_Access_or_Modification_Time)
* [Modify ACL Entries](#Modify_ACL_Entries)
* [Remove ACL Entries](#Remove_ACL_Entries)
* [Remove Default ACL](#Remove_Default_ACL)
* [Remove ACL](#Remove_ACL)
* [Set ACL](#Set_ACL)
* [Get ACL Status](#Get_ACL_Status)
* [Check access](#Check_access)
* [Get all Storage Policies](#Get_all_Storage_Policies)
* [Set Storage Policy](#Set_Storage_Policy)
* [Unset Storage Policy](#Unset_Storage_Policy)
* [Get Storage Policy](#Get_Storage_Policy)
* [Extended Attributes(XAttrs) Operations](#Extended_AttributesXAttrs_Operations)
* [Set XAttr](#Set_XAttr)
* [Remove XAttr](#Remove_XAttr)
* [Get an XAttr](#Get_an_XAttr)
* [Get multiple XAttrs](#Get_multiple_XAttrs)
* [Get all XAttrs](#Get_all_XAttrs)
* [List all XAttrs](#List_all_XAttrs)
* [Snapshot Operations](#Snapshot_Operations)
* [Create Snapshot](#Create_Snapshot)
* [Delete Snapshot](#Delete_Snapshot)
* [Rename Snapshot](#Rename_Snapshot)
* [Delegation Token Operations](#Delegation_Token_Operations)
* [Get Delegation Token](#Get_Delegation_Token)
* [Renew Delegation Token](#Renew_Delegation_Token)
* [Cancel Delegation Token](#Cancel_Delegation_Token)
* [Error Responses](#Error_Responses)
* [HTTP Response Codes](#HTTP_Response_Codes)
* [Illegal Argument Exception](#Illegal_Argument_Exception)
* [Security Exception](#Security_Exception)
* [Access Control Exception](#Access_Control_Exception)
* [File Not Found Exception](#File_Not_Found_Exception)
* [JSON Schemas](#JSON_Schemas)
* [ACL Status JSON Schema](#ACL_Status_JSON_Schema)
* [XAttrs JSON Schema](#XAttrs_JSON_Schema)
* [XAttrNames JSON Schema](#XAttrNames_JSON_Schema)
* [Boolean JSON Schema](#Boolean_JSON_Schema)
* [ContentSummary JSON Schema](#ContentSummary_JSON_Schema)
* [FileChecksum JSON Schema](#FileChecksum_JSON_Schema)
* [FileStatus JSON Schema](#FileStatus_JSON_Schema)
* [FileStatus Properties](#FileStatus_Properties)
* [FileStatuses JSON Schema](#FileStatuses_JSON_Schema)
* [Long JSON Schema](#Long_JSON_Schema)
* [Path JSON Schema](#Path_JSON_Schema)
* [RemoteException JSON Schema](#RemoteException_JSON_Schema)
* [Token JSON Schema](#Token_JSON_Schema)
* [Token Properties](#Token_Properties)
* [BlockStoragePolicy JSON Schema](#BlockStoragePolicy_JSON_Schema)
* [BlockStoragePolicy Properties](#BlockStoragePolicy_Properties)
* [BlockStoragePolicies JSON Schema](#BlockStoragePolicies_JSON_Schema)
* [BlockLocation JSON Schema](#BlockLocation_JSON_Schema)
* [BlockLocation Properties](#BlockLocation_Properties)
* [BlockLocations JSON Schema](#BlockLocations_JSON_Schema)
* [HTTP Query Parameter Dictionary](#HTTP_Query_Parameter_Dictionary)
* [ACL Spec](#ACL_Spec)
* [XAttr Name](#XAttr_Name)
* [XAttr Value](#XAttr_Value)
* [XAttr set flag](#XAttr_set_flag)
* [XAttr value encoding](#XAttr_value_encoding)
* [Access Time](#Access_Time)
* [Block Size](#Block_Size)
* [Buffer Size](#Buffer_Size)
* [Create Flag](#Create_Flag)
* [Create Parent](#Create_Parent)
* [Delegation](#Delegation)
* [Destination](#Destination)
* [Do As](#Do_As)
* [Fs Action](#Fs_Action)
* [Group](#Group)
* [Length](#Length)
* [Modification Time](#Modification_Time)
* [Offset](#Offset)
* [Old Snapshot Name](#Old_Snapshot_Name)
* [Op](#Op)
* [Overwrite](#Overwrite)
* [Owner](#Owner)
* [Permission](#Permission)
* [Recursive](#Recursive)
* [Renewer](#Renewer)
* [Replication](#Replication)
* [Snapshot Name](#Snapshot_Name)
* [Sources](#Sources)
* [Token](#Token)
* [Token Kind](#Token_Kind)
* [Token Service](#Token_Service)
* [Username](#Username)
* [NoRedirect](#NoRedirect)
* [Storage Policy](#Storage_Policy)
* [Start After](#Start_After)
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
Document Conventions
--------------------