2023-02-14 00:55:33 -05:00
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<!--
2023-03-25 02:49:13 -04:00
| Generated by Apache Maven Doxia at 2023-03-25
2023-02-14 00:55:33 -05:00
| Rendered using Apache Maven Stylus Skin 1.5
-->
< html xmlns = "http://www.w3.org/1999/xhtml" >
< head >
< title > HttpFS – Hadoop HDFS over HTTP - Server Setup< / title >
< style type = "text/css" media = "all" >
@import url("./css/maven-base.css");
@import url("./css/maven-theme.css");
@import url("./css/site.css");
< / style >
< link rel = "stylesheet" href = "./css/print.css" type = "text/css" media = "print" / >
2023-03-25 02:49:13 -04:00
< meta name = "Date-Revision-yyyymmdd" content = "20230325" / >
2023-02-14 00:55:33 -05:00
< meta http-equiv = "Content-Type" content = "text/html; charset=UTF-8" / >
< / head >
< body class = "composite" >
< div id = "banner" >
< a href = "http://hadoop.apache.org/" id = "bannerLeft" >
< img src = "http://hadoop.apache.org/images/hadoop-logo.jpg" alt = "" / >
< / a >
< a href = "http://www.apache.org/" id = "bannerRight" >
< img src = "http://www.apache.org/images/asf_logo_wide.png" alt = "" / >
< / a >
< div class = "clear" >
< hr / >
< / div >
< / div >
< div id = "breadcrumbs" >
< div class = "xright" > < a href = "http://wiki.apache.org/hadoop" class = "externalLink" > Wiki< / a >
|
< a href = "https://gitbox.apache.org/repos/asf/hadoop.git" class = "externalLink" > git< / a >
|
< a href = "http://hadoop.apache.org/" class = "externalLink" > Apache Hadoop< / a >
2023-03-25 02:49:13 -04:00
| Last Published: 2023-03-25
2023-02-14 00:55:33 -05:00
| Version: 3.4.0-SNAPSHOT
< / div >
< div class = "clear" >
< hr / >
< / div >
< / div >
< div id = "leftColumn" >
< div id = "navcolumn" >
< h5 > General< / h5 >
< ul >
< li class = "none" >
< a href = "../index.html" > Overview< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/SingleCluster.html" > Single Node Setup< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/ClusterSetup.html" > Cluster Setup< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/CommandsManual.html" > Commands Reference< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/FileSystemShell.html" > FileSystem Shell< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/Compatibility.html" > Compatibility Specification< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/DownstreamDev.html" > Downstream Developer's Guide< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/AdminCompatibilityGuide.html" > Admin Compatibility Guide< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/InterfaceClassification.html" > Interface Classification< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/filesystem/index.html" > FileSystem Specification< / a >
< / li >
< / ul >
< h5 > Common< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/CLIMiniCluster.html" > CLI Mini Cluster< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/FairCallQueue.html" > Fair Call Queue< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/NativeLibraries.html" > Native Libraries< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/Superusers.html" > Proxy User< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/RackAwareness.html" > Rack Awareness< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/SecureMode.html" > Secure Mode< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html" > Service Level Authorization< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/HttpAuthentication.html" > HTTP Authentication< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/CredentialProviderAPI.html" > Credential Provider API< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-kms/index.html" > Hadoop KMS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/Tracing.html" > Tracing< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/UnixShellGuide.html" > Unix Shell Guide< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/registry/index.html" > Registry< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/AsyncProfilerServlet.html" > Async Profiler< / a >
< / li >
< / ul >
< h5 > HDFS< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html" > Architecture< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html" > User Guide< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HDFSCommands.html" > Commands Reference< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html" > NameNode HA With QJM< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html" > NameNode HA With NFS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/ObserverNameNode.html" > Observer NameNode< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/Federation.html" > Federation< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/ViewFs.html" > ViewFs< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/ViewFsOverloadScheme.html" > ViewFsOverloadScheme< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html" > Snapshots< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html" > Edits Viewer< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html" > Image Viewer< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html" > Permissions and HDFS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html" > Quotas and HDFS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/LibHdfs.html" > libhdfs (C API)< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/WebHDFS.html" > WebHDFS (REST API)< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-hdfs-httpfs/index.html" > HttpFS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html" > Short Circuit Local Reads< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html" > Centralized Cache Management< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html" > NFS Gateway< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html" > Rolling Upgrade< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html" > Extended Attributes< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html" > Transparent Encryption< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html" > Multihoming< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html" > Storage Policies< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/MemoryStorage.html" > Memory Storage Support< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/SLGUserGuide.html" > Synthetic Load Generator< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html" > Erasure Coding< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HDFSDiskbalancer.html" > Disk Balancer< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsUpgradeDomain.html" > Upgrade Domain< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsDataNodeAdminGuide.html" > DataNode Admin< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html" > Router Federation< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/HdfsProvidedStorage.html" > Provided Storage< / a >
< / li >
< / ul >
< h5 > MapReduce< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html" > Tutorial< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html" > Commands Reference< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html" > Compatibility with 1.x< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html" > Encrypted Shuffle< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html" > Pluggable Shuffle/Sort< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html" > Distributed Cache Deploy< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/SharedCacheSupport.html" > Support for YARN Shared Cache< / a >
< / li >
< / ul >
< h5 > MapReduce REST APIs< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredAppMasterRest.html" > MR Application Master< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html" > MR History Server< / a >
< / li >
< / ul >
< h5 > YARN< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/YARN.html" > Architecture< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/YarnCommands.html" > Commands Reference< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html" > Capacity Scheduler< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/FairScheduler.html" > Fair Scheduler< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html" > ResourceManager Restart< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html" > ResourceManager HA< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/ResourceModel.html" > Resource Model< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/NodeLabel.html" > Node Labels< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/NodeAttributes.html" > Node Attributes< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html" > Web Application Proxy< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/TimelineServer.html" > Timeline Server< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html" > Timeline Service V.2< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html" > Writing YARN Applications< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html" > YARN Application Security< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/NodeManager.html" > NodeManager< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/DockerContainers.html" > Running Applications in Docker Containers< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/RuncContainers.html" > Running Applications in runC Containers< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html" > Using CGroups< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/SecureContainer.html" > Secure Containers< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/ReservationSystem.html" > Reservation System< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html" > Graceful Decommission< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html" > Opportunistic Containers< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/Federation.html" > YARN Federation< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/SharedCache.html" > Shared Cache< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/UsingGpus.html" > Using GPU< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/UsingFPGA.html" > Using FPGA< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html" > Placement Constraints< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/YarnUI2.html" > YARN UI2< / a >
< / li >
< / ul >
< h5 > YARN REST APIs< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html" > Introduction< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html" > Resource Manager< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html" > Node Manager< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/TimelineServer.html#Timeline_Server_REST_API_v1" > Timeline Server< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html#Timeline_Service_v.2_REST_API" > Timeline Service V.2< / a >
< / li >
< / ul >
< h5 > YARN Service< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/yarn-service/Overview.html" > Overview< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/yarn-service/QuickStart.html" > QuickStart< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/yarn-service/Concepts.html" > Concepts< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html" > Yarn Service API< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/yarn-service/ServiceDiscovery.html" > Service Discovery< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-site/yarn-service/SystemServices.html" > System Services< / a >
< / li >
< / ul >
< h5 > Hadoop Compatible File Systems< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-aliyun/tools/hadoop-aliyun/index.html" > Aliyun OSS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-aws/tools/hadoop-aws/index.html" > Amazon S3< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-azure/index.html" > Azure Blob Storage< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-azure-datalake/index.html" > Azure Data Lake Storage< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-cos/cloud-storage/index.html" > Tencent COS< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-huaweicloud/cloud-storage/index.html" > Huaweicloud OBS< / a >
< / li >
< / ul >
< h5 > Auth< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-auth/index.html" > Overview< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-auth/Examples.html" > Examples< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-auth/Configuration.html" > Configuration< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-auth/BuildingIt.html" > Building< / a >
< / li >
< / ul >
< h5 > Tools< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-streaming/HadoopStreaming.html" > Hadoop Streaming< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-archives/HadoopArchives.html" > Hadoop Archives< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-archive-logs/HadoopArchiveLogs.html" > Hadoop Archive Logs< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-distcp/DistCp.html" > DistCp< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-federation-balance/HDFSFederationBalance.html" > HDFS Federation Balance< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-gridmix/GridMix.html" > GridMix< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-rumen/Rumen.html" > Rumen< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-resourceestimator/ResourceEstimator.html" > Resource Estimator Service< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-sls/SchedulerLoadSimulator.html" > Scheduler Load Simulator< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/Benchmarking.html" > Hadoop Benchmarking< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-dynamometer/Dynamometer.html" > Dynamometer< / a >
< / li >
< / ul >
< h5 > Reference< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/release/" > Changelog and Release Notes< / a >
< / li >
< li class = "none" >
< a href = "../api/index.html" > Java API docs< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/UnixShellAPI.html" > Unix Shell API< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/Metrics.html" > Metrics< / a >
< / li >
< / ul >
< h5 > Configuration< / h5 >
< ul >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/core-default.xml" > core-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml" > hdfs-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-hdfs-rbf/hdfs-rbf-default.xml" > hdfs-rbf-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml" > mapred-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-yarn/hadoop-yarn-common/yarn-default.xml" > yarn-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-kms/kms-default.html" > kms-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-hdfs-httpfs/httpfs-default.html" > httpfs-default.xml< / a >
< / li >
< li class = "none" >
< a href = "../hadoop-project-dist/hadoop-common/DeprecatedProperties.html" > Deprecated Properties< / a >
< / li >
< / ul >
< a href = "http://maven.apache.org/" title = "Built by Maven" class = "poweredBy" >
< img alt = "Built by Maven" src = "./images/logos/maven-feather.png" / >
< / a >
< / div >
< / div >
< div id = "bodyColumn" >
< div id = "contentBox" >
<!-- -
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
< h1 > Hadoop HDFS over HTTP - Server Setup< / h1 >
< p > This page explains how to quickly setup HttpFS with Pseudo authentication against a Hadoop cluster with Pseudo authentication.< / p > < section >
< h2 > < a name = "Install_HttpFS" > < / a > Install HttpFS< / h2 >
< div class = "source" >
< div class = "source" >
< pre > ~ $ tar xzf httpfs-3.4.0-SNAPSHOT.tar.gz
< / pre > < / div > < / div >
< / section > < section >
< h2 > < a name = "Configure_HttpFS" > < / a > Configure HttpFS< / h2 >
< p > By default, HttpFS assumes that Hadoop configuration files (< code > core-site.xml & hdfs-site.xml< / code > ) are in the HttpFS configuration directory.< / p >
< p > If this is not the case, add to the < code > httpfs-site.xml< / code > file the < code > httpfs.hadoop.config.dir< / code > property set to the location of the Hadoop configuration directory.< / p > < / section > < section >
< h2 > < a name = "Configure_Hadoop" > < / a > Configure Hadoop< / h2 >
< p > Edit Hadoop < code > core-site.xml< / code > and defined the Unix user that will run the HttpFS server as a proxyuser. For example:< / p >
< div class = "source" >
< div class = "source" >
< pre > < property>
< name> hadoop.proxyuser.#HTTPFSUSER#.hosts< /name>
< value> httpfs-host.foo.com< /value>
< /property>
< property>
< name> hadoop.proxyuser.#HTTPFSUSER#.groups< /name>
< value> *< /value>
< /property>
< / pre > < / div > < / div >
< p > IMPORTANT: Replace < code > #HTTPFSUSER#< / code > with the Unix user that will start the HttpFS server.< / p > < / section > < section >
< h2 > < a name = "Restart_Hadoop" > < / a > Restart Hadoop< / h2 >
< p > You need to restart Hadoop for the proxyuser configuration to become active.< / p > < / section > < section >
< h2 > < a name = "Start.2FStop_HttpFS" > < / a > Start/Stop HttpFS< / h2 >
< p > To start/stop HttpFS, use < code > hdfs --daemon start|stop httpfs< / code > . For example:< / p >
< div class = "source" >
< div class = "source" >
< pre > hadoop-3.4.0-SNAPSHOT $ hdfs --daemon start httpfs
< / pre > < / div > < / div >
< p > NOTE: The script < code > httpfs.sh< / code > is deprecated. It is now just a wrapper of < code > hdfs httpfs< / code > .< / p > < / section > < section >
< h2 > < a name = "Test_HttpFS_is_working" > < / a > Test HttpFS is working< / h2 >
< div class = "source" >
< div class = "source" >
< pre > $ curl -sS 'http://< HTTPFSHOSTNAME> :14000/webhdfs/v1?op=gethomedirectory& user.name=hdfs'
{" Path" :" \/user\/hdfs" }
< / pre > < / div > < / div >
< / section > < section >
< h2 > < a name = "HttpFS_Configuration" > < / a > HttpFS Configuration< / h2 >
< p > HttpFS preconfigures the HTTP port to 14000.< / p >
< p > HttpFS supports the following < a href = "./httpfs-default.html" > configuration properties< / a > in the HttpFS’ s < code > etc/hadoop/httpfs-site.xml< / code > configuration file.< / p > < / section > < section >
< h2 > < a name = "HttpFS_over_HTTPS_.28SSL.29" > < / a > HttpFS over HTTPS (SSL)< / h2 >
< p > Enable SSL in < code > etc/hadoop/httpfs-site.xml< / code > :< / p >
< div class = "source" >
< div class = "source" >
< pre > < property>
< name> httpfs.ssl.enabled< /name>
< value> true< /value>
< description>
Whether SSL is enabled. Default is false, i.e. disabled.
< /description>
< /property>
< / pre > < / div > < / div >
< p > Configure < code > etc/hadoop/ssl-server.xml< / code > with proper values, for example:< / p >
< div class = "source" >
< div class = "source" >
< pre > < property>
< name> ssl.server.keystore.location< /name>
< value> ${user.home}/.keystore< /value>
< description> Keystore to be used. Must be specified.
< /description>
< /property>
< property>
< name> ssl.server.keystore.password< /name>
< value> < /value>
< description> Must be specified.< /description>
< /property>
< property>
< name> ssl.server.keystore.keypassword< /name>
< value> < /value>
< description> Must be specified.< /description>
< /property>
< / pre > < / div > < / div >
< p > The SSL passwords can be secured by a credential provider. See < a href = "../hadoop-project-dist/hadoop-common/CredentialProviderAPI.html" > Credential Provider API< / a > .< / p >
< p > You need to create an SSL certificate for the HttpFS server. As the < code > httpfs< / code > Unix user, using the Java < code > keytool< / code > command to create the SSL certificate:< / p >
< div class = "source" >
< div class = "source" >
< pre > $ keytool -genkey -alias jetty -keyalg RSA
< / pre > < / div > < / div >
< p > You will be asked a series of questions in an interactive prompt. It will create the keystore file, which will be named < b > .keystore< / b > and located in the < code > httpfs< / code > user home directory.< / p >
< p > The password you enter for “ keystore password” must match the value of the property < code > ssl.server.keystore.password< / code > set in the < code > ssl-server.xml< / code > in the configuration directory.< / p >
< p > The answer to “ What is your first and last name?” (i.e. “ CN” ) must be the hostname of the machine where the HttpFS Server will be running.< / p >
< p > Start HttpFS. It should work over HTTPS.< / p >
< p > Using the Hadoop < code > FileSystem< / code > API or the Hadoop FS shell, use the < code > swebhdfs://< / code > scheme. Make sure the JVM is picking up the truststore containing the public key of the SSL certificate if using a self-signed certificate. For more information about the client side settings, see < a href = "../hadoop-project-dist/hadoop-hdfs/WebHDFS.html#SSL_Configurations_for_SWebHDFS" > SSL Configurations for SWebHDFS< / a > .< / p >
< p > NOTE: Some old SSL clients may use weak ciphers that are not supported by the HttpFS server. It is recommended to upgrade the SSL client.< / p > < / section > < section >
< h2 > < a name = "Deprecated_Environment_Variables" > < / a > Deprecated Environment Variables< / h2 >
< p > The following environment variables are deprecated. Set the corresponding configuration properties instead.< / p >
< table border = "0" class = "bodyTable" >
< thead >
< tr class = "a" >
< th > Environment Variable < / th >
< th > Configuration Property < / th >
< th > Configuration File< / th > < / tr >
< / thead > < tbody >
< tr class = "b" >
< td > HTTPFS_HTTP_HOSTNAME < / td >
< td > httpfs.http.hostname < / td >
< td > httpfs-site.xml< / td > < / tr >
< tr class = "a" >
< td > HTTPFS_HTTP_PORT < / td >
< td > httpfs.http.port < / td >
< td > httpfs-site.xml< / td > < / tr >
< tr class = "b" >
< td > HTTPFS_MAX_HTTP_HEADER_SIZE < / td >
< td > hadoop.http.max.request.header.size and hadoop.http.max.response.header.size < / td >
< td > httpfs-site.xml< / td > < / tr >
< tr class = "a" >
< td > HTTPFS_MAX_THREADS < / td >
< td > hadoop.http.max.threads < / td >
< td > httpfs-site.xml< / td > < / tr >
< tr class = "b" >
< td > HTTPFS_SSL_ENABLED < / td >
< td > httpfs.ssl.enabled < / td >
< td > httpfs-site.xml< / td > < / tr >
< tr class = "a" >
< td > HTTPFS_SSL_KEYSTORE_FILE < / td >
< td > ssl.server.keystore.location < / td >
< td > ssl-server.xml< / td > < / tr >
< tr class = "b" >
< td > HTTPFS_SSL_KEYSTORE_PASS < / td >
< td > ssl.server.keystore.password < / td >
< td > ssl-server.xml< / td > < / tr >
< / tbody >
< / table > < / section > < section >
< h2 > < a name = "HTTP_Default_Services" > < / a > HTTP Default Services< / h2 >
< table border = "0" class = "bodyTable" >
< thead >
< tr class = "a" >
< th > Name < / th >
< th > Description< / th > < / tr >
< / thead > < tbody >
< tr class = "b" >
< td > /conf < / td >
< td > Display configuration properties< / td > < / tr >
< tr class = "a" >
< td > /jmx < / td >
< td > Java JMX management interface< / td > < / tr >
< tr class = "b" >
< td > /logLevel < / td >
< td > Get or set log level per class< / td > < / tr >
< tr class = "a" >
< td > /logs < / td >
< td > Display log files< / td > < / tr >
< tr class = "b" >
< td > /stacks < / td >
< td > Display JVM stacks< / td > < / tr >
< tr class = "a" >
< td > /static/index.html < / td >
< td > The static home page< / td > < / tr >
< tr class = "b" >
< td > /prof < / td >
< td > Async Profiler endpoint< / td > < / tr >
< / tbody >
< / table >
< p > To control the access to servlet < code > /conf< / code > , < code > /jmx< / code > , < code > /logLevel< / code > , < code > /logs< / code > , < code > /stacks< / code > and < code > /prof< / code > , configure the following properties in < code > httpfs-site.xml< / code > :< / p >
< div class = "source" >
< div class = "source" >
< pre > < property>
< name> hadoop.security.authorization< /name>
< value> true< /value>
< description> Is service-level authorization enabled?< /description>
< /property>
< property>
< name> hadoop.security.instrumentation.requires.admin< /name>
< value> true< /value>
< description>
Indicates if administrator ACLs are required to access
instrumentation servlets (JMX, METRICS, CONF, STACKS, PROF).
< /description>
< /property>
< property>
< name> httpfs.http.administrators< /name>
< value> < /value>
< description> ACL for the admins, this configuration is used to control
who can access the default servlets for HttpFS server. The value
should be a comma separated list of users and groups. The user list
comes first and is separated by a space followed by the group list,
e.g. " user1,user2 group1,group2" . Both users and groups are optional,
so " user1" , " group1" , " " , " user1 group1" , " user1,user2 group1,group2"
are all valid (note the leading space in " group1" ). '*' grants access
to all users and groups, e.g. '*', '* ' and ' *' are all valid.
< /description>
< /property>
< / pre > < / div > < / div > < / section >
< / div >
< / div >
< div class = "clear" >
< hr / >
< / div >
< div id = "footer" >
< div class = "xright" >
© 2008-2023
Apache Software Foundation
- < a href = "http://maven.apache.org/privacy-policy.html" > Privacy Policy< / a > .
Apache Maven, Maven, Apache, the Apache feather logo, and the Apache Maven project logos are trademarks of The Apache Software Foundation.
< / div >
< div class = "clear" >
< hr / >
< / div >
< / div >
< / body >
< / html >
