1026 lines
53 KiB
HTML
1026 lines
53 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<!--
|
|
| Generated by Apache Maven Doxia at 2023-03-05
|
|
| Rendered using Apache Maven Stylus Skin 1.5
|
|
-->
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<title>Apache Hadoop 3.4.0-SNAPSHOT – class org.apache.hadoop.fs.FSDataInputStream</title>
|
|
<style type="text/css" media="all">
|
|
@import url("../css/maven-base.css");
|
|
@import url("../css/maven-theme.css");
|
|
@import url("../css/site.css");
|
|
</style>
|
|
<link rel="stylesheet" href="../css/print.css" type="text/css" media="print" />
|
|
<meta name="Date-Revision-yyyymmdd" content="20230305" />
|
|
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
|
|
</head>
|
|
<body class="composite">
|
|
<div id="banner">
|
|
<a href="http://hadoop.apache.org/" id="bannerLeft">
|
|
<img src="http://hadoop.apache.org/images/hadoop-logo.jpg" alt="" />
|
|
</a>
|
|
<a href="http://www.apache.org/" id="bannerRight">
|
|
<img src="http://www.apache.org/images/asf_logo_wide.png" alt="" />
|
|
</a>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
</div>
|
|
<div id="breadcrumbs">
|
|
|
|
<div class="xright"> <a href="http://wiki.apache.org/hadoop" class="externalLink">Wiki</a>
|
|
|
|
|
<a href="https://gitbox.apache.org/repos/asf/hadoop.git" class="externalLink">git</a>
|
|
|
|
|
<a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
|
|
|
|
| Last Published: 2023-03-05
|
|
| Version: 3.4.0-SNAPSHOT
|
|
</div>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
</div>
|
|
<div id="leftColumn">
|
|
<div id="navcolumn">
|
|
|
|
<h5>General</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../index.html">Overview</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/SingleCluster.html">Single Node Setup</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/ClusterSetup.html">Cluster Setup</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/CommandsManual.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/FileSystemShell.html">FileSystem Shell</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/Compatibility.html">Compatibility Specification</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/DownstreamDev.html">Downstream Developer's Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/AdminCompatibilityGuide.html">Admin Compatibility Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/InterfaceClassification.html">Interface Classification</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/filesystem/index.html">FileSystem Specification</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Common</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/CLIMiniCluster.html">CLI Mini Cluster</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/FairCallQueue.html">Fair Call Queue</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/NativeLibraries.html">Native Libraries</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/Superusers.html">Proxy User</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/RackAwareness.html">Rack Awareness</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/SecureMode.html">Secure Mode</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html">Service Level Authorization</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/HttpAuthentication.html">HTTP Authentication</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/CredentialProviderAPI.html">Credential Provider API</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-kms/index.html">Hadoop KMS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/Tracing.html">Tracing</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/UnixShellGuide.html">Unix Shell Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/registry/index.html">Registry</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/AsyncProfilerServlet.html">Async Profiler</a>
|
|
</li>
|
|
</ul>
|
|
<h5>HDFS</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html">Architecture</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html">User Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HDFSCommands.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html">NameNode HA With QJM</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html">NameNode HA With NFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/ObserverNameNode.html">Observer NameNode</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/Federation.html">Federation</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/ViewFs.html">ViewFs</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/ViewFsOverloadScheme.html">ViewFsOverloadScheme</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html">Snapshots</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html">Edits Viewer</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html">Image Viewer</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html">Permissions and HDFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html">Quotas and HDFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/LibHdfs.html">libhdfs (C API)</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/WebHDFS.html">WebHDFS (REST API)</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-hdfs-httpfs/index.html">HttpFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Short Circuit Local Reads</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html">Centralized Cache Management</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html">NFS Gateway</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html">Rolling Upgrade</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html">Extended Attributes</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html">Transparent Encryption</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html">Multihoming</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html">Storage Policies</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/MemoryStorage.html">Memory Storage Support</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/SLGUserGuide.html">Synthetic Load Generator</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html">Erasure Coding</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HDFSDiskbalancer.html">Disk Balancer</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsUpgradeDomain.html">Upgrade Domain</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsDataNodeAdminGuide.html">DataNode Admin</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html">Router Federation</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/HdfsProvidedStorage.html">Provided Storage</a>
|
|
</li>
|
|
</ul>
|
|
<h5>MapReduce</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html">Tutorial</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html">Compatibility with 1.x</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html">Encrypted Shuffle</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html">Pluggable Shuffle/Sort</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html">Distributed Cache Deploy</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/SharedCacheSupport.html">Support for YARN Shared Cache</a>
|
|
</li>
|
|
</ul>
|
|
<h5>MapReduce REST APIs</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredAppMasterRest.html">MR Application Master</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html">MR History Server</a>
|
|
</li>
|
|
</ul>
|
|
<h5>YARN</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/YARN.html">Architecture</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/YarnCommands.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html">Capacity Scheduler</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/FairScheduler.html">Fair Scheduler</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html">ResourceManager Restart</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html">ResourceManager HA</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/ResourceModel.html">Resource Model</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/NodeLabel.html">Node Labels</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/NodeAttributes.html">Node Attributes</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html">Web Application Proxy</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html">Timeline Server</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html">Timeline Service V.2</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html">Writing YARN Applications</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html">YARN Application Security</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/NodeManager.html">NodeManager</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/DockerContainers.html">Running Applications in Docker Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/RuncContainers.html">Running Applications in runC Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html">Using CGroups</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/SecureContainer.html">Secure Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/ReservationSystem.html">Reservation System</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html">Graceful Decommission</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html">Opportunistic Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/Federation.html">YARN Federation</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/SharedCache.html">Shared Cache</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/UsingGpus.html">Using GPU</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/UsingFPGA.html">Using FPGA</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html">Placement Constraints</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/YarnUI2.html">YARN UI2</a>
|
|
</li>
|
|
</ul>
|
|
<h5>YARN REST APIs</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html">Introduction</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">Resource Manager</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html">Node Manager</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html#Timeline_Server_REST_API_v1">Timeline Server</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html#Timeline_Service_v.2_REST_API">Timeline Service V.2</a>
|
|
</li>
|
|
</ul>
|
|
<h5>YARN Service</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/yarn-service/Overview.html">Overview</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/yarn-service/QuickStart.html">QuickStart</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/yarn-service/Concepts.html">Concepts</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html">Yarn Service API</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/yarn-service/ServiceDiscovery.html">Service Discovery</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-site/yarn-service/SystemServices.html">System Services</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Hadoop Compatible File Systems</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-aliyun/tools/hadoop-aliyun/index.html">Aliyun OSS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-aws/tools/hadoop-aws/index.html">Amazon S3</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-azure/index.html">Azure Blob Storage</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-azure-datalake/index.html">Azure Data Lake Storage</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-cos/cloud-storage/index.html">Tencent COS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-huaweicloud/cloud-storage/index.html">Huaweicloud OBS</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Auth</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-auth/index.html">Overview</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-auth/Examples.html">Examples</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-auth/Configuration.html">Configuration</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-auth/BuildingIt.html">Building</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Tools</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-streaming/HadoopStreaming.html">Hadoop Streaming</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-archives/HadoopArchives.html">Hadoop Archives</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-archive-logs/HadoopArchiveLogs.html">Hadoop Archive Logs</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-distcp/DistCp.html">DistCp</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-federation-balance/HDFSFederationBalance.html">HDFS Federation Balance</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-gridmix/GridMix.html">GridMix</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-rumen/Rumen.html">Rumen</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-resourceestimator/ResourceEstimator.html">Resource Estimator Service</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-sls/SchedulerLoadSimulator.html">Scheduler Load Simulator</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/Benchmarking.html">Hadoop Benchmarking</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-dynamometer/Dynamometer.html">Dynamometer</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Reference</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/release/">Changelog and Release Notes</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../api/index.html">Java API docs</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/UnixShellAPI.html">Unix Shell API</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/Metrics.html">Metrics</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Configuration</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/core-default.xml">core-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml">hdfs-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-hdfs-rbf/hdfs-rbf-default.xml">hdfs-rbf-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml">mapred-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-yarn/hadoop-yarn-common/yarn-default.xml">yarn-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-kms/kms-default.html">kms-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-hdfs-httpfs/httpfs-default.html">httpfs-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../hadoop-project-dist/hadoop-common/DeprecatedProperties.html">Deprecated Properties</a>
|
|
</li>
|
|
</ul>
|
|
<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
|
|
<img alt="Built by Maven" src="../images/logos/maven-feather.png"/>
|
|
</a>
|
|
|
|
</div>
|
|
</div>
|
|
<div id="bodyColumn">
|
|
<div id="contentBox">
|
|
<!---
|
|
Licensed under the Apache License, Version 2.0 (the "License");
|
|
you may not use this file except in compliance with the License.
|
|
You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License. See accompanying LICENSE file.
|
|
-->
|
|
<!-- ============================================================= -->
|
|
<!-- CLASS: FSDataInputStream -->
|
|
<!-- ============================================================= -->
|
|
<h1>class <code>org.apache.hadoop.fs.FSDataInputStream</code></h1>
|
|
<ul>
|
|
<li><a href="#Class_FSDataInputStream_extends_DataInputStream">Class FSDataInputStream extends DataInputStream</a>
|
|
<ul>
|
|
<li><a href="#Closeable.close.28.29">Closeable.close()</a></li>
|
|
<li><a href="#Seekable.getPos.28.29">Seekable.getPos()</a></li>
|
|
<li><a href="#InputStream.read.28.29"> InputStream.read()</a></li>
|
|
<li><a href="#InputStream.read.28buffer.5B.5D.2C_offset.2C_length.29"> InputStream.read(buffer[], offset, length)</a></li>
|
|
<li><a href="#Seekable.seek.28s.29">Seekable.seek(s)</a></li>
|
|
<li><a href="#Seekable.seekToNewSource.28offset.29">Seekable.seekToNewSource(offset)</a></li>
|
|
<li><a href="#CanUnbuffer.unbuffer.28.29">CanUnbuffer.unbuffer()</a></li></ul></li>
|
|
<li><a href="#interface_PositionedReadable"> interface PositionedReadable</a>
|
|
<ul>
|
|
<li><a href="#Implementation_preconditions">Implementation preconditions</a></li>
|
|
<li><a href="#Failure_states">Failure states</a></li>
|
|
<li><a href="#int_PositionedReadable.read.28position.2C_buffer.2C_offset.2C_length.29">int PositionedReadable.read(position, buffer, offset, length)</a></li>
|
|
<li><a href="#void_PositionedReadable.readFully.28position.2C_buffer.2C_offset.2C_length.29">void PositionedReadable.readFully(position, buffer, offset, length)</a></li>
|
|
<li><a href="#PositionedReadable.readFully.28position.2C_buffer.29">PositionedReadable.readFully(position, buffer)</a></li>
|
|
<li><a href="#default_void_readVectored.28List.3C.3F_extends_FileRange.3E_ranges.2C_IntFunction.3CByteBuffer.3E_allocate.29">default void readVectored(List<? extends FileRange> ranges, IntFunction<ByteBuffer> allocate)</a></li>
|
|
<li><a href="#minSeekForVectorReads.28.29">minSeekForVectorReads()</a></li>
|
|
<li><a href="#maxReadSizeForVectorReads.28.29">maxReadSizeForVectorReads()</a></li></ul></li>
|
|
<li><a href="#Consistency">Consistency</a></li></ul>
|
|
<section>
|
|
<h2><a name="Class_FSDataInputStream_extends_DataInputStream"></a>Class <code>FSDataInputStream extends DataInputStream</code></h2>
|
|
<p>The core behavior of <code>FSDataInputStream</code> is defined by <code>java.io.DataInputStream</code>, with extensions that add key assumptions to the system.</p>
|
|
<ol style="list-style-type: decimal">
|
|
|
|
<li>The source is a local or remote filesystem.</li>
|
|
<li>The stream being read references a finite array of bytes.</li>
|
|
<li>The length of the data does not change during the read process.</li>
|
|
<li>The contents of the data does not change during the process.</li>
|
|
<li>The source file remains present during the read process.</li>
|
|
<li>Callers may use <code>Seekable.seek()</code> to offsets within the array of bytes, with future reads starting at this offset.</li>
|
|
<li>The cost of forward and backward seeks is low.</li>
|
|
<li>There is no requirement for the stream implementation to be thread-safe.</li>
|
|
<li>BUT, if a stream implements <a href="#PositionedReadable">PositionedReadable</a>, “positioned reads” MUST be thread-safe.</li>
|
|
</ol>
|
|
<p>Files are opened via <code>FileSystem.open(p)</code>, which, if successful, returns:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>result = FSDataInputStream(0, FS.Files[p])
|
|
</pre></div></div>
|
|
|
|
<p>The stream can be modeled as:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS = (pos, data[], isOpen)
|
|
</pre></div></div>
|
|
|
|
<p>with access functions:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>pos(FSDIS)
|
|
data(FSDIS)
|
|
isOpen(FSDIS)
|
|
</pre></div></div>
|
|
|
|
<p><b>Implicit invariant</b>: the size of the data stream equals the size of the file as returned by <code>FileSystem.getFileStatus(Path p)</code></p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>forall p in dom(FS.Files[p]) :
|
|
len(data(FSDIS)) == FS.getFileStatus(p).length
|
|
</pre></div></div>
|
|
<section>
|
|
<h3><a name="Closeable.close.28.29"></a><code>Closeable.close()</code></h3>
|
|
<p>The semantics of <code>java.io.Closeable</code> are defined in the interface definition within the JRE.</p>
|
|
<p>The operation MUST be idempotent; the following sequence is not an error:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS.close();
|
|
FSDIS.close();
|
|
</pre></div></div>
|
|
<section>
|
|
<h4><a name="Implementation_Notes"></a>Implementation Notes</h4>
|
|
<ul>
|
|
|
|
<li>
|
|
|
|
<p>Implementations SHOULD be robust against failure. If an inner stream is closed, it should be checked for being <code>null</code> first.</p>
|
|
</li>
|
|
<li>
|
|
|
|
<p>Implementations SHOULD NOT raise <code>IOException</code> exceptions (or any other exception) during this operation. Client applications often ignore these, or may fail unexpectedly.</p>
|
|
</li>
|
|
</ul></section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS' = ((undefined), (undefined), False)
|
|
</pre></div></div>
|
|
</section></section><section>
|
|
<h3><a name="Seekable.getPos.28.29"></a><a name="Seekable.getPos"></a><code>Seekable.getPos()</code></h3>
|
|
<p>Return the current position. The outcome when a stream is closed is undefined.</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>isOpen(FSDIS)
|
|
</pre></div></div>
|
|
</section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>result = pos(FSDIS)
|
|
</pre></div></div>
|
|
</section></section><section>
|
|
<h3><a name="InputStream.read.28.29"></a><a name="InputStream.read"></a> <code>InputStream.read()</code></h3>
|
|
<p>Return the data at the current position.</p>
|
|
<ol style="list-style-type: decimal">
|
|
|
|
<li>Implementations should fail when a stream is closed.</li>
|
|
<li>There is no limit on how long <code>read()</code> may take to complete.</li>
|
|
</ol><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>isOpen(FSDIS)
|
|
</pre></div></div>
|
|
</section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>if ( pos < len(data) ):
|
|
FSDIS' = (pos + 1, data, True)
|
|
result = data[pos]
|
|
else
|
|
result = -1
|
|
</pre></div></div>
|
|
</section></section><section>
|
|
<h3><a name="InputStream.read.28buffer.5B.5D.2C_offset.2C_length.29"></a><a name="InputStream.read.buffer"></a> <code>InputStream.read(buffer[], offset, length)</code></h3>
|
|
<p>Read <code>length</code> bytes of data into the destination buffer, starting at offset <code>offset</code>. The source of the data is the current position of the stream, as implicitly set in <code>pos</code>.</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>isOpen(FSDIS)
|
|
buffer != null else raise NullPointerException
|
|
length >= 0
|
|
offset < len(buffer)
|
|
length <= len(buffer) - offset
|
|
pos >= 0 else raise EOFException, IOException
|
|
</pre></div></div>
|
|
|
|
<p>Exceptions that may be raised on precondition failure are</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>InvalidArgumentException
|
|
ArrayIndexOutOfBoundsException
|
|
RuntimeException
|
|
</pre></div></div>
|
|
|
|
<p>Not all filesystems check the <code>isOpen</code> state.</p></section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>if length == 0 :
|
|
result = 0
|
|
|
|
else if pos > len(data):
|
|
result = -1
|
|
|
|
else
|
|
let l = min(length, len(data)-length) :
|
|
buffer' = buffer where forall i in [0..l-1]:
|
|
buffer'[o+i] = data[pos+i]
|
|
FSDIS' = (pos+l, data, true)
|
|
result = l
|
|
</pre></div></div>
|
|
|
|
<p>The <code>java.io</code> API states that if the amount of data to be read (i.e. <code>length</code>) then the call must block until the amount of data available is greater than zero —that is, until there is some data. The call is not required to return when the buffer is full, or indeed block until there is no data left in the stream.</p>
|
|
<p>That is, rather than <code>l</code> being simply defined as <code>min(length, len(data)-length)</code>, it strictly is an integer in the range <code>1..min(length, len(data)-length)</code>. While the caller may expect as much of the buffer as possible to be filled in, it is within the specification for an implementation to always return a smaller number, perhaps only ever 1 byte.</p>
|
|
<p>What is critical is that unless the destination buffer size is 0, the call must block until at least one byte is returned. Thus, for any data source of length greater than zero, repeated invocations of this <code>read()</code> operation will eventually read all the data.</p></section></section><section>
|
|
<h3><a name="Seekable.seek.28s.29"></a><a name="Seekable.seek"></a><code>Seekable.seek(s)</code></h3><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
<p>Not all subclasses implement the Seek operation:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>supported(FSDIS, Seekable.seek) else raise [UnsupportedOperationException, IOException]
|
|
</pre></div></div>
|
|
|
|
<p>If the operation is supported, the file SHOULD be open:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>isOpen(FSDIS)
|
|
</pre></div></div>
|
|
|
|
<p>Some filesystems do not perform this check, relying on the <code>read()</code> contract to reject reads on a closed stream (e.g. <code>RawLocalFileSystem</code>).</p>
|
|
<p>A <code>seek(0)</code> MUST always succeed, as the seek position must be positive and less than the length of the Stream:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>s > 0 and ((s==0) or ((s < len(data)))) else raise [EOFException, IOException]
|
|
</pre></div></div>
|
|
|
|
<p>Some FileSystems do not raise an exception if this condition is not met. They instead return -1 on any <code>read()</code> operation where, at the time of the read, <code>len(data(FSDIS)) < pos(FSDIS)</code>.</p>
|
|
<p>After a failed seek, the value of <code>pos(FSDIS)</code> may change. As an example, seeking past the EOF may move the read position to the end of the file, <i>as well as raising an <code>EOFException</code>.</i></p></section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS' = (s, data, True)
|
|
</pre></div></div>
|
|
|
|
<p>There is an implicit invariant: a seek to the current position is a no-op</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>seek(getPos())
|
|
</pre></div></div>
|
|
|
|
<p>Implementations may recognise this operation and bypass all other precondition checks, leaving the input stream unchanged.</p>
|
|
<p>The most recent connectors to object stores all implement some form of “lazy-seek”: the <code>seek()</code> call may appear to update the stream, and the value of <code>getPos()</code> is updated, but the file is not opened/reopenend until data is actually read. Implementations of lazy seek MUST still validate the new seek position against the known length of the file. However the state of the file (i.e. does it exist, what its current length is) does not need to be refreshed at this point. The fact that a file has been deleted or truncated may not surface until that <code>read()</code> call.</p></section></section><section>
|
|
<h3><a name="Seekable.seekToNewSource.28offset.29"></a><code>Seekable.seekToNewSource(offset)</code></h3>
|
|
<p>This operation instructs the source to retrieve <code>data[]</code> from a different source from the current source. This is only relevant if the filesystem supports multiple replicas of a file and there is more than 1 replica of the data at offset <code>offset</code>.</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
<p>Not all subclasses implement this operation, and instead either raise an exception or return <code>False</code>.</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>supported(FSDIS, Seekable.seekToNewSource) else raise [UnsupportedOperationException, IOException]
|
|
</pre></div></div>
|
|
|
|
<p>Examples: <code>CompressionInputStream</code> , <code>HttpFSFileSystem</code></p>
|
|
<p>If supported, the file must be open:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>isOpen(FSDIS)
|
|
</pre></div></div>
|
|
</section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
<p>The majority of subclasses that do not implement this operation simply fail.</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>if not supported(FSDIS, Seekable.seekToNewSource(s)):
|
|
result = False
|
|
</pre></div></div>
|
|
|
|
<p>Examples: <code>RawLocalFileSystem</code> , <code>HttpFSFileSystem</code></p>
|
|
<p>If the operation is supported and there is a new location for the data:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre> FSDIS' = (pos, data', true)
|
|
result = True
|
|
</pre></div></div>
|
|
|
|
<p>The new data is the original data (or an updated version of it, as covered in the Consistency section below), but the block containing the data at <code>offset</code> is sourced from a different replica.</p>
|
|
<p>If there is no other copy, <code>FSDIS</code> is not updated; the response indicates this:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre> result = False
|
|
</pre></div></div>
|
|
|
|
<p>Outside of test methods, the primary use of this method is in the {{FSInputChecker}} class, which can react to a checksum error in a read by attempting to source the data elsewhere. If a new source can be found it attempts to reread and recheck that portion of the file.</p></section></section><section>
|
|
<h3><a name="CanUnbuffer.unbuffer.28.29"></a><code>CanUnbuffer.unbuffer()</code></h3>
|
|
<p>This operation instructs the source to release any system resources they are currently holding on to, such as buffers, sockets, file descriptors, etc. Any subsequent IO operation will likely have to reacquire these resources. Unbuffering is useful in situation where streams need to remain open, but no IO operation is expected from the stream in the immediate future (examples include file handle cacheing).</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
<p>Not all subclasses implement this operation. In addition to implementing <code>CanUnbuffer</code>. Subclasses must implement the <code>StreamCapabilities</code> interface and <code>StreamCapabilities.hasCapability(UNBUFFER)</code> must return true. If a subclass implements <code>CanUnbuffer</code> but does not report the functionality via <code>StreamCapabilities</code> then the call to <code>unbuffer</code> does nothing. If a subclass reports that it does implement <code>UNBUFFER</code>, but does not implement the <code>CanUnbuffer</code> interface, an <code>UnsupportedOperationException</code> is thrown.</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>supported(FSDIS, StreamCapabilities.hasCapability && FSDIS.hasCapability(UNBUFFER) && CanUnbuffer.unbuffer)
|
|
</pre></div></div>
|
|
|
|
<p>This method is not thread-safe. If <code>unbuffer</code> is called while a <code>read</code> is in progress, the outcome is undefined.</p>
|
|
<p><code>unbuffer</code> can be called on a closed file, in which case <code>unbuffer</code> will do nothing.</p></section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
<p>The majority of subclasses that do not implement this operation simply do nothing.</p>
|
|
<p>If the operation is supported, <code>unbuffer</code> releases any and all system resources associated with the stream. The exact list of what these resources are is generally implementation dependent, however, in general, it may include buffers, sockets, file descriptors, etc.</p></section></section></section><section>
|
|
<h2><a name="interface_PositionedReadable"></a><a name="PositionedReadable"></a> interface <code>PositionedReadable</code></h2>
|
|
<p>The <code>PositionedReadable</code> operations supply “positioned reads” (“pread”). They provide the ability to read data into a buffer from a specific position in the data stream. Positioned reads equate to a <a href="#Seekable.seek"><code>Seekable.seek</code></a> at a particular offset followed by a <a href="#InputStream.read.buffer"><code>InputStream.read(buffer[], offset, length)</code></a>, only there is a single method invocation, rather than <code>seek</code> then <code>read</code>, and two positioned reads can <i>optionally</i> run concurrently over a single instance of a <code>FSDataInputStream</code> stream.</p>
|
|
<p>The interface declares positioned reads thread-safe (some of the implementations do not follow this guarantee).</p>
|
|
<p>Any positional read run concurrent with a stream operation — e.g. <a href="#Seekable.seek"><code>Seekable.seek</code></a>, <a href="#Seekable.getPos"><code>Seekable.getPos()</code></a>, and <a href="#InputStream.read"><code>InputStream.read()</code></a> — MUST run in isolation; there must not be mutual interference.</p>
|
|
<p>Concurrent positional reads and stream operations MUST be serializable; one may block the other so they run in series but, for better throughput and ‘liveness’, they SHOULD run concurrently.</p>
|
|
<p>Given two parallel positional reads, one at <code>pos1</code> for <code>len1</code> into buffer <code>dest1</code>, and another at <code>pos2</code> for <code>len2</code> into buffer <code>dest2</code>, AND given a concurrent, stream read run after a seek to <code>pos3</code>, the resultant buffers MUST be filled as follows, even if the reads happen to overlap on the underlying stream:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>// Positioned read #1
|
|
read(pos1, dest1, ... len1) -> dest1[0..len1 - 1] =
|
|
[data(FS, path, pos1), data(FS, path, pos1 + 1) ... data(FS, path, pos1 + len1 - 1]
|
|
|
|
// Positioned read #2
|
|
read(pos2, dest2, ... len2) -> dest2[0..len2 - 1] =
|
|
[data(FS, path, pos2), data(FS, path, pos2 + 1) ... data(FS, path, pos2 + len2 - 1]
|
|
|
|
// Stream read
|
|
seek(pos3);
|
|
read(dest3, ... len3) -> dest3[0..len3 - 1] =
|
|
[data(FS, path, pos3), data(FS, path, pos3 + 1) ... data(FS, path, pos3 + len3 - 1]
|
|
</pre></div></div>
|
|
|
|
<p>Note that implementations are not required to be atomic; the intermediate state of the operation (the change in the value of <code>getPos()</code>) may be visible.</p><section>
|
|
<h3><a name="Implementation_preconditions"></a>Implementation preconditions</h3>
|
|
<p>Not all <code>FSDataInputStream</code> implementations support these operations. Those that do not implement <code>Seekable.seek()</code> do not implement the <code>PositionedReadable</code> interface.</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>supported(FSDIS, Seekable.seek) else raise [UnsupportedOperationException, IOException]
|
|
</pre></div></div>
|
|
|
|
<p>This could be considered obvious: if a stream is not <code>Seekable</code>, a client cannot seek to a location. It is also a side effect of the base class implementation, which uses <code>Seekable.seek()</code>.</p>
|
|
<p><b>Implicit invariant</b>: for all <code>PositionedReadable</code> operations, the value of <code>pos</code> is unchanged at the end of the operation</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>pos(FSDIS') == pos(FSDIS)
|
|
</pre></div></div>
|
|
</section><section>
|
|
<h3><a name="Failure_states"></a>Failure states</h3>
|
|
<p>For any operations that fail, the contents of the destination <code>buffer</code> are undefined. Implementations may overwrite part or all of the buffer before reporting a failure.</p></section><section>
|
|
<h3><a name="int_PositionedReadable.read.28position.2C_buffer.2C_offset.2C_length.29"></a><code>int PositionedReadable.read(position, buffer, offset, length)</code></h3>
|
|
<p>Read as much data as possible into the buffer space allocated for it.</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>position >= 0 else raise [EOFException, IOException, IllegalArgumentException, RuntimeException]
|
|
len(buffer) - offset >= length else raise [IndexOutOfBoundException, RuntimeException]
|
|
length >= 0
|
|
offset >= 0
|
|
</pre></div></div>
|
|
</section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
<p>The amount of data read is the less of the length or the amount of data available from the specified position:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>let available = min(length, len(data)-position)
|
|
buffer'[offset..(offset+available-1)] = data[position..position+available -1]
|
|
result = available
|
|
</pre></div></div>
|
|
|
|
<ol style="list-style-type: decimal">
|
|
|
|
<li>A return value of -1 means that the stream had no more available data.</li>
|
|
<li>An invocation with <code>length==0</code> implicitly does not read any data; implementations may short-cut the operation and omit any IO. In such instances, checks for the stream being at the end of the file may be omitted.</li>
|
|
<li>If an IO exception occurs during the read operation(s), the final state of <code>buffer</code> is undefined.</li>
|
|
</ol></section></section><section>
|
|
<h3><a name="void_PositionedReadable.readFully.28position.2C_buffer.2C_offset.2C_length.29"></a><code>void PositionedReadable.readFully(position, buffer, offset, length)</code></h3>
|
|
<p>Read exactly <code>length</code> bytes of data into the buffer, failing if there is not enough data available.</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>position >= 0 else raise [EOFException, IOException, IllegalArgumentException, RuntimeException]
|
|
length >= 0
|
|
offset >= 0
|
|
len(buffer) - offset >= length else raise [IndexOutOfBoundException, RuntimeException]
|
|
(position + length) <= len(data) else raise [EOFException, IOException]
|
|
</pre></div></div>
|
|
|
|
<p>If an IO exception occurs during the read operation(s), the final state of <code>buffer</code> is undefined.</p>
|
|
<p>If there is not enough data in the input stream to satisfy the requests, the final state of <code>buffer</code> is undefined.</p></section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
<p>The buffer from offset <code>offset</code> is filled with the data starting at <code>position</code></p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>buffer'[offset..(offset+length-1)] = data[position..(position + length -1)]
|
|
</pre></div></div>
|
|
</section></section><section>
|
|
<h3><a name="PositionedReadable.readFully.28position.2C_buffer.29"></a><code>PositionedReadable.readFully(position, buffer)</code></h3>
|
|
<p>The semantics of this are exactly equivalent to</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>readFully(position, buffer, 0, len(buffer))
|
|
</pre></div></div>
|
|
|
|
<p>That is, the buffer is filled entirely with the contents of the input source from position <code>position</code></p></section><section>
|
|
<h3><a name="default_void_readVectored.28List.3C.3F_extends_FileRange.3E_ranges.2C_IntFunction.3CByteBuffer.3E_allocate.29"></a><code>default void readVectored(List<? extends FileRange> ranges, IntFunction<ByteBuffer> allocate)</code></h3>
|
|
<p>Read fully data for a list of ranges asynchronously. The default implementation iterates through the ranges, tries to coalesce the ranges based on values of <code>minSeekForVectorReads</code> and <code>maxReadSizeForVectorReads</code> and then read each merged ranges synchronously, but the intent is sub classes can implement efficient implementation. Reading in both direct and heap byte buffers are supported. Also, clients are encouraged to use <code>WeakReferencedElasticByteBufferPool</code> for allocating buffers such that even direct buffers are garbage collected when they are no longer referenced.</p>
|
|
<p>The position returned by <code>getPos()</code> after <code>readVectored()</code> is undefined.</p>
|
|
<p>If a file is changed while the <code>readVectored()</code> operation is in progress, the output is undefined. Some ranges may have old data, some may have new, and some may have both.</p>
|
|
<p>While a <code>readVectored()</code> operation is in progress, normal read api calls may block.</p>
|
|
<p>Note: Don’t use direct buffers for reading from ChecksumFileSystem as that may lead to memory fragmentation explained in HADOOP-18296.</p><section>
|
|
<h4><a name="Preconditions"></a>Preconditions</h4>
|
|
<p>For each requested range:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>range.getOffset >= 0 else raise IllegalArgumentException
|
|
range.getLength >= 0 else raise EOFException
|
|
</pre></div></div>
|
|
</section><section>
|
|
<h4><a name="Postconditions"></a>Postconditions</h4>
|
|
<p>For each requested range:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>range.getData() returns CompletableFuture<ByteBuffer> which will have data
|
|
from range.getOffset to range.getLength.
|
|
</pre></div></div>
|
|
</section></section><section>
|
|
<h3><a name="minSeekForVectorReads.28.29"></a><code>minSeekForVectorReads()</code></h3>
|
|
<p>The smallest reasonable seek. Two ranges won’t be merged together if the difference between end of first and start of next range is more than this value.</p></section><section>
|
|
<h3><a name="maxReadSizeForVectorReads.28.29"></a><code>maxReadSizeForVectorReads()</code></h3>
|
|
<p>Maximum number of bytes which can be read in one go after merging the ranges. Two ranges won’t be merged if the combined data to be read is more than this value. Essentially setting this to 0 will disable the merging of ranges.</p></section></section><section>
|
|
<h2><a name="Consistency"></a>Consistency</h2>
|
|
<ul>
|
|
|
|
<li>All readers, local and remote, of a data stream FSDIS provided from a <code>FileSystem.open(p)</code> are expected to receive access to the data of <code>FS.Files[p]</code> at the time of opening.</li>
|
|
<li>If the underlying data is changed during the read process, these changes MAY or MAY NOT be visible.</li>
|
|
<li>Such changes that are visible MAY be partially visible.</li>
|
|
</ul>
|
|
<p>At time t0</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS0 = FS'read(p) = (0, data0[])
|
|
</pre></div></div>
|
|
|
|
<p>At time t1</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FS' = FS' where FS'.Files[p] = data1
|
|
</pre></div></div>
|
|
|
|
<p>From time <code>t >= t1</code>, the value of <code>FSDIS0</code> is undefined.</p>
|
|
<p>It may be unchanged</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS0.data == data0
|
|
|
|
forall l in len(FSDIS0.data):
|
|
FSDIS0.read() == data0[l]
|
|
</pre></div></div>
|
|
|
|
<p>It may pick up the new data</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>FSDIS0.data == data1
|
|
|
|
forall l in len(FSDIS0.data):
|
|
FSDIS0.read() == data1[l]
|
|
</pre></div></div>
|
|
|
|
<p>It may be inconsistent, such that a read of an offset returns data from either of the datasets</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>forall l in len(FSDIS0.data):
|
|
(FSDIS0.read(l) == data0[l]) or (FSDIS0.read(l) == data1[l]))
|
|
</pre></div></div>
|
|
|
|
<p>That is, every value read may be from the original or updated file.</p>
|
|
<p>It may also be inconsistent on repeated reads of same offset, that is at time <code>t2 > t1</code>:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>r2 = FSDIS0.read(l)
|
|
</pre></div></div>
|
|
|
|
<p>While at time <code>t3 > t2</code>:</p>
|
|
|
|
<div class="source">
|
|
<div class="source">
|
|
<pre>r3 = FSDIS0.read(l)
|
|
</pre></div></div>
|
|
|
|
<p>It may be that <code>r3 != r2</code>. (That is, some of the data my be cached or replicated, and on a subsequent read, a different version of the file’s contents are returned).</p>
|
|
<p>Similarly, if the data at the path <code>p</code>, is deleted, this change MAY or MAY not be visible during read operations performed on <code>FSDIS0</code>.</p></section>
|
|
</div>
|
|
</div>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
<div id="footer">
|
|
<div class="xright">
|
|
© 2008-2023
|
|
Apache Software Foundation
|
|
|
|
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a>.
|
|
Apache Maven, Maven, Apache, the Apache feather logo, and the Apache Maven project logos are trademarks of The Apache Software Foundation.
|
|
</div>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
</div>
|
|
</body>
|
|
</html>
|