Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
hadoop/hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html

890 lines
39 KiB
HTML
Raw Blame History

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!--
| Generated by Apache Maven Doxia at 2023-05-16
| Rendered using Apache Maven Stylus Skin 1.5
-->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Apache Hadoop 3.4.0-SNAPSHOT &#x2013; HDFS Snapshots</title>
<style type="text/css" media="all">
@import url("./css/maven-base.css");
@import url("./css/maven-theme.css");
@import url("./css/site.css");
</style>
<link rel="stylesheet" href="./css/print.css" type="text/css" media="print" />
<meta name="Date-Revision-yyyymmdd" content="20230516" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body class="composite">
<div id="banner">
<a href="http://hadoop.apache.org/" id="bannerLeft">
<img src="http://hadoop.apache.org/images/hadoop-logo.jpg" alt="" />
</a>
<a href="http://www.apache.org/" id="bannerRight">
<img src="http://www.apache.org/images/asf_logo_wide.png" alt="" />
</a>
<div class="clear">
<hr/>
</div>
</div>
<div id="breadcrumbs">
<div class="xright"> <a href="http://wiki.apache.org/hadoop" class="externalLink">Wiki</a>
|
<a href="https://gitbox.apache.org/repos/asf/hadoop.git" class="externalLink">git</a>
|
<a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
&nbsp;| Last Published: 2023-05-16
&nbsp;| Version: 3.4.0-SNAPSHOT
</div>
<div class="clear">
<hr/>
</div>
</div>
<div id="leftColumn">
<div id="navcolumn">
<h5>General</h5>
<ul>
<li class="none">
<a href="../../index.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/SingleCluster.html">Single Node Setup</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/ClusterSetup.html">Cluster Setup</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CommandsManual.html">Commands Reference</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/FileSystemShell.html">FileSystem Shell</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Compatibility.html">Compatibility Specification</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/DownstreamDev.html">Downstream Developer's Guide</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/AdminCompatibilityGuide.html">Admin Compatibility Guide</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/InterfaceClassification.html">Interface Classification</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/filesystem/index.html">FileSystem Specification</a>
</li>
</ul>
<h5>Common</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CLIMiniCluster.html">CLI Mini Cluster</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/FairCallQueue.html">Fair Call Queue</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/NativeLibraries.html">Native Libraries</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Superusers.html">Proxy User</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/RackAwareness.html">Rack Awareness</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/SecureMode.html">Secure Mode</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html">Service Level Authorization</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/HttpAuthentication.html">HTTP Authentication</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/CredentialProviderAPI.html">Credential Provider API</a>
</li>
<li class="none">
<a href="../../hadoop-kms/index.html">Hadoop KMS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Tracing.html">Tracing</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/UnixShellGuide.html">Unix Shell Guide</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/registry/index.html">Registry</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/AsyncProfilerServlet.html">Async Profiler</a>
</li>
</ul>
<h5>HDFS</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html">Architecture</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html">User Guide</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HDFSCommands.html">Commands Reference</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html">NameNode HA With QJM</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html">NameNode HA With NFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ObserverNameNode.html">Observer NameNode</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/Federation.html">Federation</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ViewFs.html">ViewFs</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ViewFsOverloadScheme.html">ViewFsOverloadScheme</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html">Snapshots</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html">Edits Viewer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html">Image Viewer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html">Permissions and HDFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html">Quotas and HDFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/LibHdfs.html">libhdfs (C API)</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/WebHDFS.html">WebHDFS (REST API)</a>
</li>
<li class="none">
<a href="../../hadoop-hdfs-httpfs/index.html">HttpFS</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Short Circuit Local Reads</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html">Centralized Cache Management</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html">NFS Gateway</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html">Rolling Upgrade</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html">Extended Attributes</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html">Transparent Encryption</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html">Multihoming</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html">Storage Policies</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/MemoryStorage.html">Memory Storage Support</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/SLGUserGuide.html">Synthetic Load Generator</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html">Erasure Coding</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HDFSDiskbalancer.html">Disk Balancer</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsUpgradeDomain.html">Upgrade Domain</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsDataNodeAdminGuide.html">DataNode Admin</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html">Router Federation</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/HdfsProvidedStorage.html">Provided Storage</a>
</li>
</ul>
<h5>MapReduce</h5>
<ul>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html">Tutorial</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html">Commands Reference</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html">Compatibility with 1.x</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html">Encrypted Shuffle</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html">Pluggable Shuffle/Sort</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html">Distributed Cache Deploy</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/SharedCacheSupport.html">Support for YARN Shared Cache</a>
</li>
</ul>
<h5>MapReduce REST APIs</h5>
<ul>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredAppMasterRest.html">MR Application Master</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html">MR History Server</a>
</li>
</ul>
<h5>YARN</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YARN.html">Architecture</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YarnCommands.html">Commands Reference</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html">Capacity Scheduler</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/FairScheduler.html">Fair Scheduler</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html">ResourceManager Restart</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html">ResourceManager HA</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ResourceModel.html">Resource Model</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeLabel.html">Node Labels</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeAttributes.html">Node Attributes</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html">Web Application Proxy</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html">Timeline Server</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html">Timeline Service V.2</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html">Writing YARN Applications</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html">YARN Application Security</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeManager.html">NodeManager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/DockerContainers.html">Running Applications in Docker Containers</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/RuncContainers.html">Running Applications in runC Containers</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html">Using CGroups</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/SecureContainer.html">Secure Containers</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ReservationSystem.html">Reservation System</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html">Graceful Decommission</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html">Opportunistic Containers</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/Federation.html">YARN Federation</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/SharedCache.html">Shared Cache</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/UsingGpus.html">Using GPU</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/UsingFPGA.html">Using FPGA</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html">Placement Constraints</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/YarnUI2.html">YARN UI2</a>
</li>
</ul>
<h5>YARN REST APIs</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html">Introduction</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">Resource Manager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html">Node Manager</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html#Timeline_Server_REST_API_v1">Timeline Server</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html#Timeline_Service_v.2_REST_API">Timeline Service V.2</a>
</li>
</ul>
<h5>YARN Service</h5>
<ul>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/Overview.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/QuickStart.html">QuickStart</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/Concepts.html">Concepts</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html">Yarn Service API</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/ServiceDiscovery.html">Service Discovery</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/SystemServices.html">System Services</a>
</li>
</ul>
<h5>Hadoop Compatible File Systems</h5>
<ul>
<li class="none">
<a href="../../hadoop-aliyun/tools/hadoop-aliyun/index.html">Aliyun OSS</a>
</li>
<li class="none">
<a href="../../hadoop-aws/tools/hadoop-aws/index.html">Amazon S3</a>
</li>
<li class="none">
<a href="../../hadoop-azure/index.html">Azure Blob Storage</a>
</li>
<li class="none">
<a href="../../hadoop-azure-datalake/index.html">Azure Data Lake Storage</a>
</li>
<li class="none">
<a href="../../hadoop-cos/cloud-storage/index.html">Tencent COS</a>
</li>
<li class="none">
<a href="../../hadoop-huaweicloud/cloud-storage/index.html">Huaweicloud OBS</a>
</li>
</ul>
<h5>Auth</h5>
<ul>
<li class="none">
<a href="../../hadoop-auth/index.html">Overview</a>
</li>
<li class="none">
<a href="../../hadoop-auth/Examples.html">Examples</a>
</li>
<li class="none">
<a href="../../hadoop-auth/Configuration.html">Configuration</a>
</li>
<li class="none">
<a href="../../hadoop-auth/BuildingIt.html">Building</a>
</li>
</ul>
<h5>Tools</h5>
<ul>
<li class="none">
<a href="../../hadoop-streaming/HadoopStreaming.html">Hadoop Streaming</a>
</li>
<li class="none">
<a href="../../hadoop-archives/HadoopArchives.html">Hadoop Archives</a>
</li>
<li class="none">
<a href="../../hadoop-archive-logs/HadoopArchiveLogs.html">Hadoop Archive Logs</a>
</li>
<li class="none">
<a href="../../hadoop-distcp/DistCp.html">DistCp</a>
</li>
<li class="none">
<a href="../../hadoop-federation-balance/HDFSFederationBalance.html">HDFS Federation Balance</a>
</li>
<li class="none">
<a href="../../hadoop-gridmix/GridMix.html">GridMix</a>
</li>
<li class="none">
<a href="../../hadoop-rumen/Rumen.html">Rumen</a>
</li>
<li class="none">
<a href="../../hadoop-resourceestimator/ResourceEstimator.html">Resource Estimator Service</a>
</li>
<li class="none">
<a href="../../hadoop-sls/SchedulerLoadSimulator.html">Scheduler Load Simulator</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Benchmarking.html">Hadoop Benchmarking</a>
</li>
<li class="none">
<a href="../../hadoop-dynamometer/Dynamometer.html">Dynamometer</a>
</li>
</ul>
<h5>Reference</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/release/">Changelog and Release Notes</a>
</li>
<li class="none">
<a href="../../api/index.html">Java API docs</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/UnixShellAPI.html">Unix Shell API</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/Metrics.html">Metrics</a>
</li>
</ul>
<h5>Configuration</h5>
<ul>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/core-default.xml">core-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml">hdfs-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-hdfs-rbf/hdfs-rbf-default.xml">hdfs-rbf-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml">mapred-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-yarn/hadoop-yarn-common/yarn-default.xml">yarn-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-kms/kms-default.html">kms-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-hdfs-httpfs/httpfs-default.html">httpfs-default.xml</a>
</li>
<li class="none">
<a href="../../hadoop-project-dist/hadoop-common/DeprecatedProperties.html">Deprecated Properties</a>
</li>
</ul>
<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
<img alt="Built by Maven" src="./images/logos/maven-feather.png"/>
</a>
</div>
</div>
<div id="bodyColumn">
<div id="contentBox">
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<h1>HDFS Snapshots</h1>
<ul>
<li><a href="#Overview">Overview</a>
<ul>
<li><a href="#Snapshottable_Directories">Snapshottable Directories</a></li>
<li><a href="#Snapshot_Paths">Snapshot Paths</a></li></ul></li>
<li><a href="#Upgrading_to_a_version_of_HDFS_with_snapshots">Upgrading to a version of HDFS with snapshots</a></li>
<li><a href="#Snapshot_Operations">Snapshot Operations</a>
<ul>
<li><a href="#Administrator_Operations">Administrator Operations</a>
<ul>
<li><a href="#Allow_Snapshots">Allow Snapshots</a></li>
<li><a href="#Disallow_Snapshots">Disallow Snapshots</a></li></ul></li>
<li><a href="#User_Operations">User Operations</a>
<ul>
<li><a href="#Create_Snapshots">Create Snapshots</a></li>
<li><a href="#Delete_Snapshots">Delete Snapshots</a></li>
<li><a href="#Rename_Snapshots">Rename Snapshots</a></li>
<li><a href="#Get_Snapshottable_Directory_Listing">Get Snapshottable Directory Listing</a></li>
<li><a href="#Get_Snapshot_Listing">Get Snapshot Listing</a></li>
<li><a href="#Get_Snapshots_Difference_Report">Get Snapshots Difference Report</a></li></ul></li></ul></li></ul>
<section>
<h2><a name="Overview"></a>Overview</h2>
<p>HDFS Snapshots are read-only point-in-time copies of the file system. Snapshots can be taken on a subtree of the file system or the entire file system. Some common use cases of snapshots are data backup, protection against user errors and disaster recovery.</p>
<p>The implementation of HDFS Snapshots is efficient:</p>
<ul>
<li>
<p>Snapshot creation is instantaneous: the cost is <i>O(1)</i> excluding the inode lookup time.</p>
</li>
<li>
<p>Additional memory is used only when modifications are made relative to a snapshot: memory usage is <i>O(M)</i>, where <i>M</i> is the number of modified files/directories.</p>
</li>
<li>
<p>Blocks in datanodes are not copied: the snapshot files record the block list and the file size. There is no data copying.</p>
</li>
<li>
<p>Snapshots do not adversely affect regular HDFS operations: modifications are recorded in reverse chronological order so that the current data can be accessed directly. The snapshot data is computed by subtracting the modifications from the current data.</p>
</li>
</ul><section>
<h3><a name="Snapshottable_Directories"></a>Snapshottable Directories</h3>
<p>Snapshots can be taken on any directory once the directory has been set as <i>snapshottable</i>. A snapshottable directory is able to accommodate 65,536 simultaneous snapshots. There is no limit on the number of snapshottable directories. Administrators may set any directory to be snapshottable. If there are snapshots in a snapshottable directory, the directory can be neither deleted nor renamed before all the snapshots are deleted.</p>
<p>Nested snapshottable directories are currently not allowed. In other words, a directory cannot be set to snapshottable if one of its ancestors/descendants is a snapshottable directory.</p></section><section>
<h3><a name="Snapshot_Paths"></a>Snapshot Paths</h3>
<p>For a snapshottable directory, the path component <i>&#x201c;.snapshot&#x201d;</i> is used for accessing its snapshots. Suppose <code>/foo</code> is a snapshottable directory, <code>/foo/bar</code> is a file/directory in <code>/foo</code>, and <code>/foo</code> has a snapshot <code>s0</code>. Then, the path <code>/foo/.snapshot/s0/bar</code> refers to the snapshot copy of <code>/foo/bar</code>. The usual API and CLI can work with the &#x201c;.snapshot&#x201d; paths. The following are some examples.</p>
<ul>
<li>
<p>Listing all the snapshots under a snapshottable directory:</p>
<div class="source">
<div class="source">
<pre>hdfs dfs -ls /foo/.snapshot
</pre></div></div>
</li>
<li>
<p>Listing the files in snapshot <code>s0</code>:</p>
<div class="source">
<div class="source">
<pre>hdfs dfs -ls /foo/.snapshot/s0
</pre></div></div>
</li>
<li>
<p>Copying a file from snapshot <code>s0</code>:</p>
<div class="source">
<div class="source">
<pre>hdfs dfs -cp -ptopax /foo/.snapshot/s0/bar /tmp
</pre></div></div>
<p>Note that this example uses the preserve option to preserve timestamps, ownership, permission, ACLs and XAttrs.</p>
</li>
</ul></section></section><section>
<h2><a name="Upgrading_to_a_version_of_HDFS_with_snapshots"></a>Upgrading to a version of HDFS with snapshots</h2>
<p>The HDFS snapshot feature introduces a new reserved path name used to interact with snapshots: <code>.snapshot</code>. When upgrading from an older version of HDFS which does not support snapshots, existing paths named <code>.snapshot</code> need to first be renamed or deleted to avoid conflicting with the reserved path. See the upgrade section in <a href="HdfsUserGuide.html#Upgrade_and_Rollback">the HDFS user guide</a> for more information.</p></section><section>
<h2><a name="Snapshot_Operations"></a>Snapshot Operations</h2><section>
<h3><a name="Administrator_Operations"></a>Administrator Operations</h3>
<p>The operations described in this section require superuser privilege.</p><section>
<h4><a name="Allow_Snapshots"></a>Allow Snapshots</h4>
<p>Allowing snapshots of a directory to be created. If the operation completes successfully, the directory becomes snapshottable.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs dfsadmin -allowSnapshot &lt;path&gt;
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>See also the corresponding Java API <code>void allowSnapshot(Path path)</code> in <code>HdfsAdmin</code>.</p></section><section>
<h4><a name="Disallow_Snapshots"></a>Disallow Snapshots</h4>
<p>Disallowing snapshots of a directory to be created. All snapshots of the directory must be deleted before disallowing snapshots.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs dfsadmin -disallowSnapshot &lt;path&gt;
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>See also the corresponding Java API <code>void disallowSnapshot(Path path)</code> in <code>HdfsAdmin</code>.</p></section></section><section>
<h3><a name="User_Operations"></a>User Operations</h3>
<p>The section describes user operations. Note that HDFS superuser can perform all the operations without satisfying the permission requirement in the individual operations.</p><section>
<h4><a name="Create_Snapshots"></a>Create Snapshots</h4>
<p>Create a snapshot of a snapshottable directory. This operation requires owner privilege of the snapshottable directory.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs dfs -createSnapshot &lt;path&gt; [&lt;snapshotName&gt;]
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
<tr class="b">
<td> snapshotName </td>
<td> The snapshot name, which is an optional argument. When it is omitted, a default name is generated using a timestamp with the format <code>&quot;'s'yyyyMMdd-HHmmss.SSS&quot;</code>, e.g. <code>&quot;s20130412-151029.033&quot;</code>. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>See also the corresponding Java API <code>Path createSnapshot(Path path)</code> and <code>Path createSnapshot(Path path, String snapshotName)</code> in <a href="../../api/org/apache/hadoop/fs/FileSystem.html">FileSystem</a> The snapshot path is returned in these methods.</p></section><section>
<h4><a name="Delete_Snapshots"></a>Delete Snapshots</h4>
<p>Delete a snapshot of from a snapshottable directory. This operation requires owner privilege of the snapshottable directory.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs dfs -deleteSnapshot &lt;path&gt; &lt;snapshotName&gt;
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
<tr class="b">
<td> snapshotName </td>
<td> The snapshot name. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>See also the corresponding Java API <code>void deleteSnapshot(Path path, String snapshotName)</code> in <a href="../../api/org/apache/hadoop/fs/FileSystem.html">FileSystem</a>.</p></section><section>
<h4><a name="Rename_Snapshots"></a>Rename Snapshots</h4>
<p>Rename a snapshot. This operation requires owner privilege of the snapshottable directory.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs dfs -renameSnapshot &lt;path&gt; &lt;oldName&gt; &lt;newName&gt;
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
<tr class="b">
<td> oldName </td>
<td> The old snapshot name. </td></tr>
<tr class="a">
<td> newName </td>
<td> The new snapshot name. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>See also the corresponding Java API <code>void renameSnapshot(Path path, String oldName, String newName)</code> in <a href="../../api/org/apache/hadoop/fs/FileSystem.html">FileSystem</a>.</p></section><section>
<h4><a name="Get_Snapshottable_Directory_Listing"></a>Get Snapshottable Directory Listing</h4>
<p>Get all the snapshottable directories where the current user has permission to take snapshtos.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs lsSnapshottableDir
</pre></div></div>
</li>
<li>
<p>Arguments: none</p>
</li>
</ul>
<p>See also the corresponding Java API <code>SnapshottableDirectoryStatus[] getSnapshottableDirectoryListing()</code> in <code>DistributedFileSystem</code>.</p></section><section>
<h4><a name="Get_Snapshot_Listing"></a>Get Snapshot Listing</h4>
<p>Get all the snapshots for a snapshottable directory.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs lsSnapshot &lt;snapshotRoot&gt;
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>See also the corresponding Java API <code>SnapshotStatus[] getSnapshotListing()</code> in <code>DistributedFileSystem</code>.</p></section><section>
<h4><a name="Get_Snapshots_Difference_Report"></a>Get Snapshots Difference Report</h4>
<p>Get the differences between two snapshots. This operation requires read access privilege for all files/directories in both snapshots.</p>
<ul>
<li>
<p>Command:</p>
<div class="source">
<div class="source">
<pre>hdfs snapshotDiff &lt;path&gt; &lt;fromSnapshot&gt; &lt;toSnapshot&gt;
</pre></div></div>
</li>
<li>
<p>Arguments:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> path </td>
<td> The path of the snapshottable directory. </td></tr>
<tr class="b">
<td> fromSnapshot </td>
<td> The name of the starting snapshot. </td></tr>
<tr class="a">
<td> toSnapshot </td>
<td> The name of the ending snapshot. </td></tr>
</tbody>
</table>
<p>Note that snapshotDiff can be used to get the difference report between two snapshots, or between a snapshot and the current status of a directory. Users can use &#x201c;.&#x201d; to represent the current status.</p>
</li>
<li>
<p>Results:</p>
<table border="0" class="bodyTable">
<thead></thead><tbody>
<tr class="a">
<td> + </td>
<td> The file/directory has been created. </td></tr>
<tr class="b">
<td> - </td>
<td> The file/directory has been deleted. </td></tr>
<tr class="a">
<td> M </td>
<td> The file/directory has been modified. </td></tr>
<tr class="b">
<td> R </td>
<td> The file/directory has been renamed. </td></tr>
</tbody>
</table>
</li>
</ul>
<p>A <i>RENAME</i> entry indicates a file/directory has been renamed but is still under the same snapshottable directory. A file/directory is reported as deleted if it was renamed to outside of the snapshottble directory. A file/directory renamed from outside of the snapshottble directory is reported as newly created.</p>
<p>The snapshot difference report does not guarantee the same operation sequence. For example, if we rename the directory <i>&#x201c;/foo&#x201d;</i> to <i>&#x201c;/foo2&#x201d;</i>, and then append new data to the file <i>&#x201c;/foo2/bar&#x201d;</i>, the difference report will be:</p>
<div class="source">
<div class="source">
<pre>R. /foo -&gt; /foo2
M. /foo/bar
</pre></div></div>
<p>I.e., the changes on the files/directories under a renamed directory is reported using the original path before the rename (<i>&#x201c;/foo/bar&#x201d;</i> in the above example).</p>
<p>See also the corresponding Java API <code>SnapshotDiffReport getSnapshotDiffReport(Path path, String fromSnapshot, String toSnapshot)</code> in <code>DistributedFileSystem</code>.</p></section></section></section>
</div>
</div>
<div class="clear">
<hr/>
</div>
<div id="footer">
<div class="xright">
&#169; 2008-2023
Apache Software Foundation
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a>.
Apache Maven, Maven, Apache, the Apache feather logo, and the Apache Maven project logos are trademarks of The Apache Software Foundation.
</div>
<div class="clear">
<hr/>
</div>
</div>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink