diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md index 06b73904b0a..56a2ab8c15e 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ArchivalStorage.md @@ -15,19 +15,7 @@ Archival Storage, SSD & Memory ============================== -* [Archival Storage, SSD & Memory](#Archival_Storage_SSD__Memory) - * [Introduction](#Introduction) - * [Storage Types and Storage Policies](#Storage_Types_and_Storage_Policies) - * [Storage Types: ARCHIVE, DISK, SSD and RAM\_DISK](#Storage_Types:_ARCHIVE_DISK_SSD_and_RAM_DISK) - * [Storage Policies: Hot, Warm, Cold, All\_SSD, One\_SSD and Lazy\_Persist](#Storage_Policies:_Hot_Warm_Cold_All_SSD_One_SSD_and_Lazy_Persist) - * [Storage Policy Resolution](#Storage_Policy_Resolution) - * [Configuration](#Configuration) - * [Mover - A New Data Migration Tool](#Mover_-_A_New_Data_Migration_Tool) - * [Storage Policy Commands](#Storage_Policy_Commands) - * [List Storage Policies](#List_Storage_Policies) - * [Set Storage Policy](#Set_Storage_Policy) - * [Unset Storage Policy](#Unset_Storage_Policy) - * [Get Storage Policy](#Get_Storage_Policy) + Introduction ------------ diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md index 210d25ca364..89ad6709a37 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/CentralizedCacheManagement.md @@ -15,30 +15,7 @@ Centralized Cache Management in HDFS ==================================== -* [Overview](#Overview) -* [Use Cases](#Use_Cases) -* [Architecture](#Architecture) -* [Concepts](#Concepts) - * [Cache directive](#Cache_directive) - * [Cache pool](#Cache_pool) -* [cacheadmin command-line interface](#cacheadmin_command-line_interface) - * [Cache directive commands](#Cache_directive_commands) - * [addDirective](#addDirective) - * [removeDirective](#removeDirective) - * [removeDirectives](#removeDirectives) - * [listDirectives](#listDirectives) - * [Cache pool commands](#Cache_pool_commands) - * [addPool](#addPool) - * [modifyPool](#modifyPool) - * [removePool](#removePool) - * [listPools](#listPools) - * [help](#help) -* [Configuration](#Configuration) - * [Native Libraries](#Native_Libraries) - * [Configuration Properties](#Configuration_Properties) - * [Required](#Required) - * [Optional](#Optional) - * [OS Limits](#OS_Limits) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ExtendedAttributes.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ExtendedAttributes.md index eb527abafd7..65a3cad124e 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ExtendedAttributes.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ExtendedAttributes.md @@ -15,13 +15,7 @@ Extended Attributes in HDFS =========================== -* [Overview](#Overview) - * [HDFS extended attributes](#HDFS_extended_attributes) - * [Namespaces and Permissions](#Namespaces_and_Permissions) -* [Interacting with extended attributes](#Interacting_with_extended_attributes) - * [getfattr](#getfattr) - * [setfattr](#setfattr) -* [Configuration options](#Configuration_options) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Federation.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Federation.md index 0ed5df8aae2..ddf474ac492 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Federation.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/Federation.md @@ -15,20 +15,7 @@ HDFS Federation =============== -* [HDFS Federation](#HDFS_Federation) - * [Background](#Background) - * [Multiple Namenodes/Namespaces](#Multiple_NamenodesNamespaces) - * [Key Benefits](#Key_Benefits) - * [Federation Configuration](#Federation_Configuration) - * [Configuration:](#Configuration:) - * [Formatting Namenodes](#Formatting_Namenodes) - * [Upgrading from an older release and configuring federation](#Upgrading_from_an_older_release_and_configuring_federation) - * [Adding a new Namenode to an existing HDFS cluster](#Adding_a_new_Namenode_to_an_existing_HDFS_cluster) - * [Managing the cluster](#Managing_the_cluster) - * [Starting and stopping cluster](#Starting_and_stopping_cluster) - * [Balancer](#Balancer) - * [Decommissioning](#Decommissioning) - * [Cluster Web Console](#Cluster_Web_Console) + This guide provides an overview of the HDFS Federation feature and how to configure and manage the federated cluster. diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md index 8825820fb49..628cd01f3cd 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md @@ -15,43 +15,7 @@ HDFS Commands Guide =================== -* [Overview](#Overview) -* [User Commands](#User_Commands) - * [classpath](#classpath) - * [dfs](#dfs) - * [envvars](#envvars) - * [fetchdt](#fetchdt) - * [fsck](#fsck) - * [getconf](#getconf) - * [groups](#groups) - * [lsSnapshottableDir](#lsSnapshottableDir) - * [jmxget](#jmxget) - * [oev](#oev) - * [oiv](#oiv) - * [oiv\_legacy](#oiv_legacy) - * [snapshotDiff](#snapshotDiff) - * [version](#version) -* [Administration Commands](#Administration_Commands) - * [balancer](#balancer) - * [cacheadmin](#cacheadmin) - * [crypto](#crypto) - * [datanode](#datanode) - * [dfsadmin](#dfsadmin) - * [diskbalancer](#diskbalancer) - * [erasurecode](#erasurecode) - * [haadmin](#haadmin) - * [journalnode](#journalnode) - * [mover](#mover) - * [namenode](#namenode) - * [nfs3](#nfs3) - * [portmap](#portmap) - * [secondarynamenode](#secondarynamenode) - * [storagepolicies](#storagepolicies) - * [zkfc](#zkfc) -* [Debug Commands](#Debug_Commands) - * [verifyMeta](#verifyMeta) - * [computeMeta](#computeMeta) - * [recoverLease](#recoverLease) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSDiskbalancer.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSDiskbalancer.md index b7e55cb01cb..b6eb62e7004 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSDiskbalancer.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSDiskbalancer.md @@ -15,11 +15,7 @@ HDFS Disk Balancer =================== -* [Overview](#Overview) -* [Architecture](#Architecture) -* [Commands](#Commands) -* [Settings](#Settings) -* [Debugging](#Debugging) + Overview diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md index d5dbd0bc39f..8c504ac5e77 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSErasureCoding.md @@ -15,15 +15,7 @@ HDFS Erasure Coding =================== -* [HDFS Erasure Coding](#HDFS_Erasure_Coding) - * [Purpose](#Purpose) - * [Background](#Background) - * [Architecture](#Architecture) - * [Deployment](#Deployment) - * [Cluster and hardware configuration](#Cluster_and_hardware_configuration) - * [Configuration keys](#Configuration_keys) - * [Enable Intel ISA-L](#Enable_Intel_ISA-L) - * [Administrative commands](#Administrative_commands) + Purpose ------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md index f3ae2810a78..5caf8c27c80 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md @@ -15,29 +15,7 @@ HDFS High Availability ====================== -* [HDFS High Availability](#HDFS_High_Availability) - * [Purpose](#Purpose) - * [Note: Using the Quorum Journal Manager or Conventional Shared Storage](#Note:_Using_the_Quorum_Journal_Manager_or_Conventional_Shared_Storage) - * [Background](#Background) - * [Architecture](#Architecture) - * [Hardware resources](#Hardware_resources) - * [Deployment](#Deployment) - * [Configuration overview](#Configuration_overview) - * [Configuration details](#Configuration_details) - * [Deployment details](#Deployment_details) - * [Administrative commands](#Administrative_commands) - * [Automatic Failover](#Automatic_Failover) - * [Introduction](#Introduction) - * [Components](#Components) - * [Deploying ZooKeeper](#Deploying_ZooKeeper) - * [Before you begin](#Before_you_begin) - * [Configuring automatic failover](#Configuring_automatic_failover) - * [Initializing HA state in ZooKeeper](#Initializing_HA_state_in_ZooKeeper) - * [Starting the cluster with start-dfs.sh](#Starting_the_cluster_with_start-dfs.sh) - * [Starting the cluster manually](#Starting_the_cluster_manually) - * [Securing access to ZooKeeper](#Securing_access_to_ZooKeeper) - * [Verifying automatic failover](#Verifying_automatic_failover) - * [Automatic Failover FAQ](#Automatic_Failover_FAQ) + Purpose ------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md index 51dc91aa12a..99bc8f02475 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithQJM.md @@ -15,30 +15,7 @@ HDFS High Availability Using the Quorum Journal Manager ======================================================= -* [HDFS High Availability Using the Quorum Journal Manager](#HDFS_High_Availability_Using_the_Quorum_Journal_Manager) - * [Purpose](#Purpose) - * [Note: Using the Quorum Journal Manager or Conventional Shared Storage](#Note:_Using_the_Quorum_Journal_Manager_or_Conventional_Shared_Storage) - * [Background](#Background) - * [Architecture](#Architecture) - * [Hardware resources](#Hardware_resources) - * [Deployment](#Deployment) - * [Configuration overview](#Configuration_overview) - * [Configuration details](#Configuration_details) - * [Deployment details](#Deployment_details) - * [Administrative commands](#Administrative_commands) - * [Automatic Failover](#Automatic_Failover) - * [Introduction](#Introduction) - * [Components](#Components) - * [Deploying ZooKeeper](#Deploying_ZooKeeper) - * [Before you begin](#Before_you_begin) - * [Configuring automatic failover](#Configuring_automatic_failover) - * [Initializing HA state in ZooKeeper](#Initializing_HA_state_in_ZooKeeper) - * [Starting the cluster with start-dfs.sh](#Starting_the_cluster_with_start-dfs.sh) - * [Starting the cluster manually](#Starting_the_cluster_manually) - * [Securing access to ZooKeeper](#Securing_access_to_ZooKeeper) - * [Verifying automatic failover](#Verifying_automatic_failover) - * [Automatic Failover FAQ](#Automatic_Failover_FAQ) - * [HDFS Upgrade/Finalization/Rollback with HA Enabled](#HDFS_UpgradeFinalizationRollback_with_HA_Enabled) + Purpose ------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md index 6f547eaf1d8..bda53a9c77b 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md @@ -15,41 +15,7 @@ HDFS Architecture ================= -* [HDFS Architecture](#HDFS_Architecture) - * [Introduction](#Introduction) - * [Assumptions and Goals](#Assumptions_and_Goals) - * [Hardware Failure](#Hardware_Failure) - * [Streaming Data Access](#Streaming_Data_Access) - * [Large Data Sets](#Large_Data_Sets) - * [Simple Coherency Model](#Simple_Coherency_Model) - * ["Moving Computation is Cheaper than Moving Data"](#aMoving_Computation_is_Cheaper_than_Moving_Data) - * [Portability Across Heterogeneous Hardware and Software Platforms](#Portability_Across_Heterogeneous_Hardware_and_Software_Platforms) - * [NameNode and DataNodes](#NameNode_and_DataNodes) - * [The File System Namespace](#The_File_System_Namespace) - * [Data Replication](#Data_Replication) - * [Replica Placement: The First Baby Steps](#Replica_Placement:_The_First_Baby_Steps) - * [Replica Selection](#Replica_Selection) - * [Safemode](#Safemode) - * [The Persistence of File System Metadata](#The_Persistence_of_File_System_Metadata) - * [The Communication Protocols](#The_Communication_Protocols) - * [Robustness](#Robustness) - * [Data Disk Failure, Heartbeats and Re-Replication](#Data_Disk_Failure_Heartbeats_and_Re-Replication) - * [Cluster Rebalancing](#Cluster_Rebalancing) - * [Data Integrity](#Data_Integrity) - * [Metadata Disk Failure](#Metadata_Disk_Failure) - * [Snapshots](#Snapshots) - * [Data Organization](#Data_Organization) - * [Data Blocks](#Data_Blocks) - * [Staging](#Staging) - * [Replication Pipelining](#Replication_Pipelining) - * [Accessibility](#Accessibility) - * [FS Shell](#FS_Shell) - * [DFSAdmin](#DFSAdmin) - * [Browser Interface](#Browser_Interface) - * [Space Reclamation](#Space_Reclamation) - * [File Deletes and Undeletes](#File_Deletes_and_Undeletes) - * [Decrease Replication Factor](#Decrease_Replication_Factor) - * [References](#References) + Introduction ------------ diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md index 58f115baf99..7a46eb05068 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsEditsViewer.md @@ -15,10 +15,7 @@ Offline Edits Viewer Guide ========================== -* [Offline Edits Viewer Guide](#Offline_Edits_Viewer_Guide) - * [Overview](#Overview) - * [Usage](#Usage) - * [Case study: Hadoop cluster recovery](#Case_study:_Hadoop_cluster_recovery) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md index f55c9fda024..b677f6af480 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsImageViewer.md @@ -15,16 +15,7 @@ Offline Image Viewer Guide ========================== -* [Offline Image Viewer Guide](#Offline_Image_Viewer_Guide) - * [Overview](#Overview) - * [Usage](#Usage) - * [Web Processor](#Web_Processor) - * [XML Processor](#XML_Processor) - * [Options](#Options) - * [Analyzing Results](#Analyzing_Results) - * [oiv\_legacy Command](#oiv_legacy_Command) - * [Usage](#Usage) - * [Options](#Options) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md index 4e1d4804f4f..2d71650178d 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsMultihoming.md @@ -17,14 +17,7 @@ HDFS Support for Multihomed Networks This document is targetted to cluster administrators deploying `HDFS` in multihomed networks. Similar support for `YARN`/`MapReduce` is work in progress and will be documented when available. -* [HDFS Support for Multihomed Networks](#HDFS_Support_for_Multihomed_Networks) - * [Multihoming Background](#Multihoming_Background) - * [Fixing Hadoop Issues In Multihomed Environments](#Fixing_Hadoop_Issues_In_Multihomed_Environments) - * [Ensuring HDFS Daemons Bind All Interfaces](#Ensuring_HDFS_Daemons_Bind_All_Interfaces) - * [Clients use Hostnames when connecting to DataNodes](#Clients_use_Hostnames_when_connecting_to_DataNodes) - * [DataNodes use HostNames when connecting to other DataNodes](#DataNodes_use_HostNames_when_connecting_to_other_DataNodes) - * [Multihoming and Hadoop Security](#Multihoming_and_Hadoop_Security) - * [Hostname Lookup](#Hostname_Lookup) + Multihoming Background ---------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md index 37a2042d089..f150b8b25fc 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsNfsGateway.md @@ -15,14 +15,7 @@ HDFS NFS Gateway ================ -* [HDFS NFS Gateway](#HDFS_NFS_Gateway) - * [Overview](#Overview) - * [Configuration](#Configuration) - * [Start and stop NFS gateway service](#Start_and_stop_NFS_gateway_service) - * [Verify validity of NFS related services](#Verify_validity_of_NFS_related_services) - * [Mount the export "/"](#Mount_the_export_) - * [Allow mounts from unprivileged clients](#Allow_mounts_from_unprivileged_clients) - * [User authentication and mapping](#User_authentication_and_mapping) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsPermissionsGuide.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsPermissionsGuide.md index 7fd9b0065ea..c50253459da 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsPermissionsGuide.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsPermissionsGuide.md @@ -15,20 +15,7 @@ HDFS Permissions Guide ====================== -* [HDFS Permissions Guide](#HDFS_Permissions_Guide) - * [Overview](#Overview) - * [User Identity](#User_Identity) - * [Group Mapping](#Group_Mapping) - * [Permission Checks](#Permission_Checks) - * [Understanding the Implementation](#Understanding_the_Implementation) - * [Changes to the File System API](#Changes_to_the_File_System_API) - * [Changes to the Application Shell](#Changes_to_the_Application_Shell) - * [The Super-User](#The_Super-User) - * [The Web Server](#The_Web_Server) - * [ACLs (Access Control Lists)](#ACLs_Access_Control_Lists) - * [ACLs File System API](#ACLs_File_System_API) - * [ACLs Shell Commands](#ACLs_Shell_Commands) - * [Configuration Parameters](#Configuration_Parameters) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsQuotaAdminGuide.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsQuotaAdminGuide.md index 107fd12b9dd..461bdbff1aa 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsQuotaAdminGuide.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsQuotaAdminGuide.md @@ -15,13 +15,7 @@ HDFS Quotas Guide ================= -* [HDFS Quotas Guide](#HDFS_Quotas_Guide) - * [Overview](#Overview) - * [Name Quotas](#Name_Quotas) - * [Space Quotas](#Space_Quotas) - * [Storage Type Quotas](#Storage_Type_Quotas) - * [Administrative Commands](#Administrative_Commands) - * [Reporting Command](#Reporting_Command) + Overview -------- @@ -128,4 +122,4 @@ An an extension to the count command of the HDFS shell reports quota values and storage type quota set and the available quota remaining for each directory. If specific storage types are given after -t option, only quota and remaining quota of the types specified will be displayed. Otherwise, quota and - remaining quota of all storage types that support quota will be displayed. \ No newline at end of file + remaining quota of all storage types that support quota will be displayed. diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsRollingUpgrade.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsRollingUpgrade.md index 889d7a0a3b5..cabdd74653b 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsRollingUpgrade.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsRollingUpgrade.md @@ -18,23 +18,7 @@ HDFS Rolling Upgrade ==================== -* [Introduction](#Introduction) -* [Upgrade](#Upgrade) - * [Upgrade without Downtime](#Upgrade_without_Downtime) - * [Upgrading Non-Federated Clusters](#Upgrading_Non-Federated_Clusters) - * [Upgrading Federated Clusters](#Upgrading_Federated_Clusters) - * [Upgrade with Downtime](#Upgrade_with_Downtime) - * [Upgrading Non-HA Clusters](#Upgrading_Non-HA_Clusters) -* [Downgrade and Rollback](#Downgrade_and_Rollback) -* [Downgrade](#Downgrade) -* [Rollback](#Rollback) -* [Commands and Startup Options for Rolling Upgrade](#Commands_and_Startup_Options_for_Rolling_Upgrade) - * [DFSAdmin Commands](#DFSAdmin_Commands) - * [dfsadmin -rollingUpgrade](#dfsadmin_-rollingUpgrade) - * [dfsadmin -getDatanodeInfo](#dfsadmin_-getDatanodeInfo) - * [dfsadmin -shutdownDatanode](#dfsadmin_-shutdownDatanode) - * [NameNode Startup Options](#NameNode_Startup_Options) - * [namenode -rollingUpgrade](#namenode_-rollingUpgrade) + Introduction diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsSnapshots.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsSnapshots.md index d856e8c24e7..af55f33b264 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsSnapshots.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsSnapshots.md @@ -18,21 +18,7 @@ HDFS Snapshots ============== -* [HDFS Snapshots](#HDFS_Snapshots) - * [Overview](#Overview) - * [Snapshottable Directories](#Snapshottable_Directories) - * [Snapshot Paths](#Snapshot_Paths) - * [Upgrading to a version of HDFS with snapshots](#Upgrading_to_a_version_of_HDFS_with_snapshots) - * [Snapshot Operations](#Snapshot_Operations) - * [Administrator Operations](#Administrator_Operations) - * [Allow Snapshots](#Allow_Snapshots) - * [Disallow Snapshots](#Disallow_Snapshots) - * [User Operations](#User_Operations) - * [Create Snapshots](#Create_Snapshots) - * [Delete Snapshots](#Delete_Snapshots) - * [Rename Snapshots](#Rename_Snapshots) - * [Get Snapshottable Directory Listing](#Get_Snapshottable_Directory_Listing) - * [Get Snapshots Difference Report](#Get_Snapshots_Difference_Report) + Overview diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md index 16bae5eee34..7a06c4131a3 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md @@ -15,28 +15,7 @@ HDFS Users Guide ================ -* [HDFS Users Guide](#HDFS_Users_Guide) - * [Purpose](#Purpose) - * [Overview](#Overview) - * [Prerequisites](#Prerequisites) - * [Web Interface](#Web_Interface) - * [Shell Commands](#Shell_Commands) - * [DFSAdmin Command](#DFSAdmin_Command) - * [Secondary NameNode](#Secondary_NameNode) - * [Checkpoint Node](#Checkpoint_Node) - * [Backup Node](#Backup_Node) - * [Import Checkpoint](#Import_Checkpoint) - * [Balancer](#Balancer) - * [Rack Awareness](#Rack_Awareness) - * [Safemode](#Safemode) - * [fsck](#fsck) - * [fetchdt](#fetchdt) - * [Recovery Mode](#Recovery_Mode) - * [Upgrade and Rollback](#Upgrade_and_Rollback) - * [DataNode Hot Swap Drive](#DataNode_Hot_Swap_Drive) - * [File Permissions and Security](#File_Permissions_and_Security) - * [Scalability](#Scalability) - * [Related Documentation](#Related_Documentation) + Purpose ------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/LibHdfs.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/LibHdfs.md index 2697cde82d3..7049dcbce2a 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/LibHdfs.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/LibHdfs.md @@ -15,13 +15,7 @@ C API libhdfs ============= -* [C API libhdfs](#C_API_libhdfs) - * [Overview](#Overview) - * [The APIs](#The_APIs) - * [A Sample Program](#A_Sample_Program) - * [How To Link With The Library](#How_To_Link_With_The_Library) - * [Common Problems](#Common_Problems) - * [Thread Safe](#Thread_Safe) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/MemoryStorage.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/MemoryStorage.md index 1ed4b5ac26f..8773860e574 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/MemoryStorage.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/MemoryStorage.md @@ -15,19 +15,7 @@ Memory Storage Support in HDFS ============================== -* [Introduction](#Introduction) -* [Administrator Configuration](#Administrator_Configuration) - * [Limit RAM used for replicas in Memory](#Limit_RAM_used_for_replicas_in_Memory) - * [Setup RAM Disks on Data Nodes](#Setup_RAM_Disks_on_Data_Nodes) - * [Choosing `tmpfs` \(vs `ramfs`\)](#Choosing_`tmpfs`_\(vs_`ramfs`\)) - * [Mount RAM Disks](#Mount_RAM_Disks) - * [Tag `tmpfs` volume with the RAM\_DISK Storage Type](#Tag_`tmpfs`_volume_with_the_RAM\_DISK_Storage_Type) - * [Ensure Storage Policies are enabled](#Ensure_Storage_Policies_are_enabled) -* [Application Usage](#Application_Usage) - * [Use the LAZY\_PERSIST Storage Policy](#Use_the_LAZY\_PERSIST_Storage_Policy) - * [Invoke `hdfs storagepolicies` command for directories](#Invoke_hdfs_storagepolicies_command_for_directories) - * [Call `setStoragePolicy` method for directories](#Call_`setStoragePolicy`_method_for_directories) - * [Pass `LAZY_PERSIST` `CreateFlag` for new files](#Pass_`LAZY_PERSIST`_`CreateFlag`_for_new_files) + Introduction ------------ diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/SLGUserGuide.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/SLGUserGuide.md index 38b293e32f5..a78633f9972 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/SLGUserGuide.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/SLGUserGuide.md @@ -15,12 +15,7 @@ Synthetic Load Generator Guide ============================== -* [Synthetic Load Generator Guide](#Synthetic_Load_Generator_Guide) - * [Overview](#Overview) - * [Synopsis](#Synopsis) - * [Test Space Population](#Test_Space_Population) - * [Structure Generator](#Structure_Generator) - * [Data Generator](#Data_Generator) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ShortCircuitLocalReads.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ShortCircuitLocalReads.md index 5150a3c5b4f..43dbe51c208 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ShortCircuitLocalReads.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ShortCircuitLocalReads.md @@ -15,12 +15,7 @@ HDFS Short-Circuit Local Reads ============================== -* [HDFS Short-Circuit Local Reads](#HDFS_Short-Circuit_Local_Reads) - * [Short-Circuit Local Reads](#Short-Circuit_Local_Reads) - * [Background](#Background) - * [Setup](#Setup) - * [Example Configuration](#Example_Configuration) - * [Legacy HDFS Short-Circuit Local Reads](#Legacy_HDFS_Short-Circuit_Local_Reads) + Short-Circuit Local Reads ------------------------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md index 1a403c66e8d..089d7b47011 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/TransparentEncryption.md @@ -15,32 +15,7 @@ Transparent Encryption in HDFS ============================== -* [Overview](#Overview) -* [Background](#Background) -* [Use Cases](#Use_Cases) -* [Architecture](#Architecture) - * [Overview](#Architecture_overview) - * [Accessing data within an encryption zone](#Accessing_data_within_an_encryption_zone) - * [Key Management Server, KeyProvider, EDEKs](#Key_Management_Server_KeyProvider_EDEKs) -* [Configuration](#Configuration) - * [Configuring the cluster KeyProvider](#Configuring_the_cluster_KeyProvider) - * [Selecting an encryption algorithm and codec](#Selecting_an_encryption_algorithm_and_codec) - * [Namenode configuration](#Namenode_configuration) -* [crypto command-line interface](#crypto_command-line_interface) - * [createZone](#createZone) - * [listZones](#listZones) - * [provisionTrash](#provisionTrash) - * [getFileEncryptionInfo](#getFileEncryptionInfo) -* [Example usage](#Example_usage) -* [Distcp considerations](#Distcp_considerations) - * [Running as the superuser](#Running_as_the_superuser) - * [Copying into encrypted locations](#Copying_into_encrypted_locations) -* [Rename and Trash considerations](#Rename_and_Trash_considerations) -* [Attack vectors](#Attack_vectors) - * [Hardware access exploits](#Hardware_access_exploits) - * [Root access exploits](#Root_access_exploits) - * [HDFS admin exploits](#HDFS_admin_exploits) - * [Rogue user exploits](#Rogue_user_exploits) + Overview -------- diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md index ddec9b05140..6e8ce67c1c1 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/ViewFs.md @@ -15,20 +15,7 @@ ViewFs Guide ============ -* [ViewFs Guide](#ViewFs_Guide) - * [Introduction](#Introduction) - * [The Old World (Prior to Federation)](#The_Old_World_Prior_to_Federation) - * [Single Namenode Clusters](#Single_Namenode_Clusters) - * [Pathnames Usage Patterns](#Pathnames_Usage_Patterns) - * [Pathname Usage Best Practices](#Pathname_Usage_Best_Practices) - * [New World – Federation and ViewFs](#New_World__Federation_and_ViewFs) - * [How The Clusters Look](#How_The_Clusters_Look) - * [A Global Namespace Per Cluster Using ViewFs](#A_Global_Namespace_Per_Cluster_Using_ViewFs) - * [Pathname Usage Patterns](#Pathname_Usage_Patterns) - * [Pathname Usage Best Practices](#Pathname_Usage_Best_Practices) - * [Renaming Pathnames Across Namespaces](#Renaming_Pathnames_Across_Namespaces) - * [FAQ](#FAQ) - * [Appendix: A Mount Table Configuration Example](#Appendix:_A_Mount_Table_Configuration_Example) + Introduction ------------ diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md index 50b43602919..7544c80ae6c 100644 --- a/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md +++ b/hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md @@ -15,128 +15,7 @@ WebHDFS REST API ================ -* [WebHDFS REST API](#WebHDFS_REST_API) - * [Document Conventions](#Document_Conventions) - * [Introduction](#Introduction) - * [Operations](#Operations) - * [FileSystem URIs vs HTTP URLs](#FileSystem_URIs_vs_HTTP_URLs) - * [HDFS Configuration Options](#HDFS_Configuration_Options) - * [Authentication](#Authentication) - * [Proxy Users](#Proxy_Users) - * [Cross-Site Request Forgery Prevention](#Cross-Site_Request_Forgery_Prevention) - * [WebHDFS Retry Policy](#WebHDFS_Retry_Policy) - * [File and Directory Operations](#File_and_Directory_Operations) - * [Create and Write to a File](#Create_and_Write_to_a_File) - * [Append to a File](#Append_to_a_File) - * [Concat File(s)](#Concat_Files) - * [Open and Read a File](#Open_and_Read_a_File) - * [Make a Directory](#Make_a_Directory) - * [Create a Symbolic Link](#Create_a_Symbolic_Link) - * [Rename a File/Directory](#Rename_a_FileDirectory) - * [Delete a File/Directory](#Delete_a_FileDirectory) - * [Truncate a File](#Truncate_a_File) - * [Status of a File/Directory](#Status_of_a_FileDirectory) - * [List a Directory](#List_a_Directory) - * [Iteratively List a Directory](#Iteratively_List_a_Directory) - * [Get File Block Locations](#Get_File_Block_Locations) - * [Other File System Operations](#Other_File_System_Operations) - * [Get Content Summary of a Directory](#Get_Content_Summary_of_a_Directory) - * [Get File Checksum](#Get_File_Checksum) - * [Get Home Directory](#Get_Home_Directory) - * [Get Trash Root](#Get_Trash_Root) - * [Set Permission](#Set_Permission) - * [Set Owner](#Set_Owner) - * [Set Replication Factor](#Set_Replication_Factor) - * [Set Access or Modification Time](#Set_Access_or_Modification_Time) - * [Modify ACL Entries](#Modify_ACL_Entries) - * [Remove ACL Entries](#Remove_ACL_Entries) - * [Remove Default ACL](#Remove_Default_ACL) - * [Remove ACL](#Remove_ACL) - * [Set ACL](#Set_ACL) - * [Get ACL Status](#Get_ACL_Status) - * [Check access](#Check_access) - * [Get all Storage Policies](#Get_all_Storage_Policies) - * [Set Storage Policy](#Set_Storage_Policy) - * [Unset Storage Policy](#Unset_Storage_Policy) - * [Get Storage Policy](#Get_Storage_Policy) - * [Extended Attributes(XAttrs) Operations](#Extended_AttributesXAttrs_Operations) - * [Set XAttr](#Set_XAttr) - * [Remove XAttr](#Remove_XAttr) - * [Get an XAttr](#Get_an_XAttr) - * [Get multiple XAttrs](#Get_multiple_XAttrs) - * [Get all XAttrs](#Get_all_XAttrs) - * [List all XAttrs](#List_all_XAttrs) - * [Snapshot Operations](#Snapshot_Operations) - * [Create Snapshot](#Create_Snapshot) - * [Delete Snapshot](#Delete_Snapshot) - * [Rename Snapshot](#Rename_Snapshot) - * [Delegation Token Operations](#Delegation_Token_Operations) - * [Get Delegation Token](#Get_Delegation_Token) - * [Renew Delegation Token](#Renew_Delegation_Token) - * [Cancel Delegation Token](#Cancel_Delegation_Token) - * [Error Responses](#Error_Responses) - * [HTTP Response Codes](#HTTP_Response_Codes) - * [Illegal Argument Exception](#Illegal_Argument_Exception) - * [Security Exception](#Security_Exception) - * [Access Control Exception](#Access_Control_Exception) - * [File Not Found Exception](#File_Not_Found_Exception) - * [JSON Schemas](#JSON_Schemas) - * [ACL Status JSON Schema](#ACL_Status_JSON_Schema) - * [XAttrs JSON Schema](#XAttrs_JSON_Schema) - * [XAttrNames JSON Schema](#XAttrNames_JSON_Schema) - * [Boolean JSON Schema](#Boolean_JSON_Schema) - * [ContentSummary JSON Schema](#ContentSummary_JSON_Schema) - * [FileChecksum JSON Schema](#FileChecksum_JSON_Schema) - * [FileStatus JSON Schema](#FileStatus_JSON_Schema) - * [FileStatus Properties](#FileStatus_Properties) - * [FileStatuses JSON Schema](#FileStatuses_JSON_Schema) - * [Long JSON Schema](#Long_JSON_Schema) - * [Path JSON Schema](#Path_JSON_Schema) - * [RemoteException JSON Schema](#RemoteException_JSON_Schema) - * [Token JSON Schema](#Token_JSON_Schema) - * [Token Properties](#Token_Properties) - * [BlockStoragePolicy JSON Schema](#BlockStoragePolicy_JSON_Schema) - * [BlockStoragePolicy Properties](#BlockStoragePolicy_Properties) - * [BlockStoragePolicies JSON Schema](#BlockStoragePolicies_JSON_Schema) - * [BlockLocation JSON Schema](#BlockLocation_JSON_Schema) - * [BlockLocation Properties](#BlockLocation_Properties) - * [BlockLocations JSON Schema](#BlockLocations_JSON_Schema) - * [HTTP Query Parameter Dictionary](#HTTP_Query_Parameter_Dictionary) - * [ACL Spec](#ACL_Spec) - * [XAttr Name](#XAttr_Name) - * [XAttr Value](#XAttr_Value) - * [XAttr set flag](#XAttr_set_flag) - * [XAttr value encoding](#XAttr_value_encoding) - * [Access Time](#Access_Time) - * [Block Size](#Block_Size) - * [Buffer Size](#Buffer_Size) - * [Create Flag](#Create_Flag) - * [Create Parent](#Create_Parent) - * [Delegation](#Delegation) - * [Destination](#Destination) - * [Do As](#Do_As) - * [Fs Action](#Fs_Action) - * [Group](#Group) - * [Length](#Length) - * [Modification Time](#Modification_Time) - * [Offset](#Offset) - * [Old Snapshot Name](#Old_Snapshot_Name) - * [Op](#Op) - * [Overwrite](#Overwrite) - * [Owner](#Owner) - * [Permission](#Permission) - * [Recursive](#Recursive) - * [Renewer](#Renewer) - * [Replication](#Replication) - * [Snapshot Name](#Snapshot_Name) - * [Sources](#Sources) - * [Token](#Token) - * [Token Kind](#Token_Kind) - * [Token Service](#Token_Service) - * [Username](#Username) - * [NoRedirect](#NoRedirect) - * [Storage Policy](#Storage_Policy) - * [Start After](#Start_After) + Document Conventions --------------------