970 lines
57 KiB
HTML
970 lines
57 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
|
|
<!--
|
|
| Generated by Apache Maven Doxia at 2023-03-29
|
|
| Rendered using Apache Maven Stylus Skin 1.5
|
|
-->
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<title>Apache Hadoop 3.4.0-SNAPSHOT – Apache Hadoop 0.22.0 Release Notes</title>
|
|
<style type="text/css" media="all">
|
|
@import url("../../css/maven-base.css");
|
|
@import url("../../css/maven-theme.css");
|
|
@import url("../../css/site.css");
|
|
</style>
|
|
<link rel="stylesheet" href="../../css/print.css" type="text/css" media="print" />
|
|
<meta name="Date-Revision-yyyymmdd" content="20230329" />
|
|
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
|
|
</head>
|
|
<body class="composite">
|
|
<div id="banner">
|
|
<a href="http://hadoop.apache.org/" id="bannerLeft">
|
|
<img src="http://hadoop.apache.org/images/hadoop-logo.jpg" alt="" />
|
|
</a>
|
|
<a href="http://www.apache.org/" id="bannerRight">
|
|
<img src="http://www.apache.org/images/asf_logo_wide.png" alt="" />
|
|
</a>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
</div>
|
|
<div id="breadcrumbs">
|
|
|
|
<div class="xright"> <a href="http://wiki.apache.org/hadoop" class="externalLink">Wiki</a>
|
|
|
|
|
<a href="https://gitbox.apache.org/repos/asf/hadoop.git" class="externalLink">git</a>
|
|
|
|
|
<a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a>
|
|
|
|
| Last Published: 2023-03-29
|
|
| Version: 3.4.0-SNAPSHOT
|
|
</div>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
</div>
|
|
<div id="leftColumn">
|
|
<div id="navcolumn">
|
|
|
|
<h5>General</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../index.html">Overview</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/SingleCluster.html">Single Node Setup</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/ClusterSetup.html">Cluster Setup</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/CommandsManual.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/FileSystemShell.html">FileSystem Shell</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/Compatibility.html">Compatibility Specification</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/DownstreamDev.html">Downstream Developer's Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/AdminCompatibilityGuide.html">Admin Compatibility Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/InterfaceClassification.html">Interface Classification</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/filesystem/index.html">FileSystem Specification</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Common</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/CLIMiniCluster.html">CLI Mini Cluster</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/FairCallQueue.html">Fair Call Queue</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/NativeLibraries.html">Native Libraries</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/Superusers.html">Proxy User</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/RackAwareness.html">Rack Awareness</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/SecureMode.html">Secure Mode</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html">Service Level Authorization</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/HttpAuthentication.html">HTTP Authentication</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/CredentialProviderAPI.html">Credential Provider API</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-kms/index.html">Hadoop KMS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/Tracing.html">Tracing</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/UnixShellGuide.html">Unix Shell Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/registry/index.html">Registry</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/AsyncProfilerServlet.html">Async Profiler</a>
|
|
</li>
|
|
</ul>
|
|
<h5>HDFS</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html">Architecture</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html">User Guide</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HDFSCommands.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html">NameNode HA With QJM</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html">NameNode HA With NFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/ObserverNameNode.html">Observer NameNode</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/Federation.html">Federation</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/ViewFs.html">ViewFs</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/ViewFsOverloadScheme.html">ViewFsOverloadScheme</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html">Snapshots</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html">Edits Viewer</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html">Image Viewer</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html">Permissions and HDFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html">Quotas and HDFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/LibHdfs.html">libhdfs (C API)</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/WebHDFS.html">WebHDFS (REST API)</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-hdfs-httpfs/index.html">HttpFS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Short Circuit Local Reads</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html">Centralized Cache Management</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html">NFS Gateway</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html">Rolling Upgrade</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html">Extended Attributes</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html">Transparent Encryption</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html">Multihoming</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html">Storage Policies</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/MemoryStorage.html">Memory Storage Support</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/SLGUserGuide.html">Synthetic Load Generator</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html">Erasure Coding</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HDFSDiskbalancer.html">Disk Balancer</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsUpgradeDomain.html">Upgrade Domain</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsDataNodeAdminGuide.html">DataNode Admin</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html">Router Federation</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/HdfsProvidedStorage.html">Provided Storage</a>
|
|
</li>
|
|
</ul>
|
|
<h5>MapReduce</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html">Tutorial</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html">Compatibility with 1.x</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html">Encrypted Shuffle</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html">Pluggable Shuffle/Sort</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html">Distributed Cache Deploy</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/SharedCacheSupport.html">Support for YARN Shared Cache</a>
|
|
</li>
|
|
</ul>
|
|
<h5>MapReduce REST APIs</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredAppMasterRest.html">MR Application Master</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html">MR History Server</a>
|
|
</li>
|
|
</ul>
|
|
<h5>YARN</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/YARN.html">Architecture</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/YarnCommands.html">Commands Reference</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html">Capacity Scheduler</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/FairScheduler.html">Fair Scheduler</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html">ResourceManager Restart</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html">ResourceManager HA</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/ResourceModel.html">Resource Model</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/NodeLabel.html">Node Labels</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/NodeAttributes.html">Node Attributes</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html">Web Application Proxy</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html">Timeline Server</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html">Timeline Service V.2</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html">Writing YARN Applications</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html">YARN Application Security</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/NodeManager.html">NodeManager</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/DockerContainers.html">Running Applications in Docker Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/RuncContainers.html">Running Applications in runC Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html">Using CGroups</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/SecureContainer.html">Secure Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/ReservationSystem.html">Reservation System</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html">Graceful Decommission</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html">Opportunistic Containers</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/Federation.html">YARN Federation</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/SharedCache.html">Shared Cache</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/UsingGpus.html">Using GPU</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/UsingFPGA.html">Using FPGA</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html">Placement Constraints</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/YarnUI2.html">YARN UI2</a>
|
|
</li>
|
|
</ul>
|
|
<h5>YARN REST APIs</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html">Introduction</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">Resource Manager</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html">Node Manager</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html#Timeline_Server_REST_API_v1">Timeline Server</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html#Timeline_Service_v.2_REST_API">Timeline Service V.2</a>
|
|
</li>
|
|
</ul>
|
|
<h5>YARN Service</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/yarn-service/Overview.html">Overview</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/yarn-service/QuickStart.html">QuickStart</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/yarn-service/Concepts.html">Concepts</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html">Yarn Service API</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/yarn-service/ServiceDiscovery.html">Service Discovery</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-site/yarn-service/SystemServices.html">System Services</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Hadoop Compatible File Systems</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-aliyun/tools/hadoop-aliyun/index.html">Aliyun OSS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-aws/tools/hadoop-aws/index.html">Amazon S3</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-azure/index.html">Azure Blob Storage</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-azure-datalake/index.html">Azure Data Lake Storage</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-cos/cloud-storage/index.html">Tencent COS</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-huaweicloud/cloud-storage/index.html">Huaweicloud OBS</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Auth</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-auth/index.html">Overview</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-auth/Examples.html">Examples</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-auth/Configuration.html">Configuration</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-auth/BuildingIt.html">Building</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Tools</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-streaming/HadoopStreaming.html">Hadoop Streaming</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-archives/HadoopArchives.html">Hadoop Archives</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-archive-logs/HadoopArchiveLogs.html">Hadoop Archive Logs</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-distcp/DistCp.html">DistCp</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-federation-balance/HDFSFederationBalance.html">HDFS Federation Balance</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-gridmix/GridMix.html">GridMix</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-rumen/Rumen.html">Rumen</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-resourceestimator/ResourceEstimator.html">Resource Estimator Service</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-sls/SchedulerLoadSimulator.html">Scheduler Load Simulator</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/Benchmarking.html">Hadoop Benchmarking</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-dynamometer/Dynamometer.html">Dynamometer</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Reference</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/release/">Changelog and Release Notes</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../api/index.html">Java API docs</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/UnixShellAPI.html">Unix Shell API</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/Metrics.html">Metrics</a>
|
|
</li>
|
|
</ul>
|
|
<h5>Configuration</h5>
|
|
<ul>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/core-default.xml">core-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml">hdfs-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-hdfs-rbf/hdfs-rbf-default.xml">hdfs-rbf-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml">mapred-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-yarn/hadoop-yarn-common/yarn-default.xml">yarn-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-kms/kms-default.html">kms-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-hdfs-httpfs/httpfs-default.html">httpfs-default.xml</a>
|
|
</li>
|
|
<li class="none">
|
|
<a href="../../../../hadoop-project-dist/hadoop-common/DeprecatedProperties.html">Deprecated Properties</a>
|
|
</li>
|
|
</ul>
|
|
<a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy">
|
|
<img alt="Built by Maven" src="../../images/logos/maven-feather.png"/>
|
|
</a>
|
|
|
|
</div>
|
|
</div>
|
|
<div id="bodyColumn">
|
|
<div id="contentBox">
|
|
<!---
|
|
# Licensed to the Apache Software Foundation (ASF) under one
|
|
# or more contributor license agreements. See the NOTICE file
|
|
# distributed with this work for additional information
|
|
# regarding copyright ownership. The ASF licenses this file
|
|
# to you under the Apache License, Version 2.0 (the
|
|
# "License"); you may not use this file except in compliance
|
|
# with the License. You may obtain a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
# See the License for the specific language governing permissions and
|
|
# limitations under the License.
|
|
-->
|
|
<h1>Apache Hadoop 0.22.0 Release Notes</h1>
|
|
<p>These release notes cover new developer and user-facing incompatibilities, important issues, features, and major improvements.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-478">MAPREDUCE-478</a> | <i>Minor</i> | <b>separate jvm param for mapper and reducer</b></li>
|
|
</ul>
|
|
<p>Allow map and reduce jvm parameters, environment variables and ulimit to be set separately.</p>
|
|
<p>Configuration changes: add mapred.map.child.java.opts add mapred.reduce.child.java.opts add mapred.map.child.env add mapred.reduce.child.ulimit add mapred.map.child.env add mapred.reduce.child.ulimit deprecated mapred.child.java.opts deprecated mapred.child.env deprecated mapred.child.ulimit</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6344">HADOOP-6344</a> | <i>Major</i> | <b>rm and rmr fail to correctly move the user’s files to the trash prior to deleting when they are over quota.</b></li>
|
|
</ul>
|
|
<p>Trash feature notifies user of over-quota condition rather than silently deleting files/directories; deletion can be compelled with “rm -skiptrash”.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6599">HADOOP-6599</a> | <i>Major</i> | <b>Split RPC metrics into summary and detailed metrics</b></li>
|
|
</ul>
|
|
<p>Split existing RpcMetrics into RpcMetrics and RpcDetailedMetrics. The new RpcDetailedMetrics has per method usage details and is available under context name “rpc” and record name “detailed-metrics”</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-927">MAPREDUCE-927</a> | <i>Major</i> | <b>Cleanup of task-logs should happen in TaskTracker instead of the Child</b></li>
|
|
</ul>
|
|
<p>Moved Task log cleanup into a separate thread in TaskTracker. Added configuration “mapreduce.job.userlog.retain.hours” to specify the time(in hours) for which the user-logs are to be retained after the job completion.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6730">HADOOP-6730</a> | <i>Major</i> | <b>Bug in FileContext#copy and provide base class for FileContext tests</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1707">MAPREDUCE-1707</a> | <i>Major</i> | <b>TaskRunner can get NPE in getting ugi from TaskTracker</b></li>
|
|
</ul>
|
|
<p>Fixed a bug that causes TaskRunner to get NPE in getting ugi from TaskTracker and subsequently crashes it resulting in a failing task after task-timeout period.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1680">MAPREDUCE-1680</a> | <i>Major</i> | <b>Add a metrics to track the number of heartbeats processed</b></li>
|
|
</ul>
|
|
<p>Added a metric to track number of heartbeats processed by the JobTracker.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1109">HDFS-1109</a> | <i>Major</i> | <b>HFTP and URL Encoding</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1061">HDFS-1061</a> | <i>Minor</i> | <b>Memory footprint optimization for INodeFile object.</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1079">HDFS-1079</a> | <i>Major</i> | <b>HDFS implementation should throw exceptions defined in AbstractFileSystem</b></li>
|
|
</ul>
|
|
<p>Specific exceptions are thrown from HDFS implementation and protocol per the interface defined in AbstractFileSystem. The compatibility is not affected as the applications catch IOException and will be able to handle specific exceptions that are subclasses of IOException.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1558">MAPREDUCE-1558</a> | <i>Major</i> | <b>specify correct server principal for RefreshAuthorizationPolicyProtocol and RefreshUserToGroupMappingsProtocol protocols in MRAdmin (for HADOOP-6612)</b></li>
|
|
</ul>
|
|
<p>new config: hadoop.security.service.user.name.key this setting points to the server principal for RefreshUserToGroupMappingsProtocol. The value should be either NN or JT principal depending if it is used in DFAdmin or MRAdmin. The value is set by the application. No need for default value.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-708">HDFS-708</a> | <i>Major</i> | <b>A stress-test tool for HDFS.</b></li>
|
|
</ul>
|
|
<p>Does not currently provide anything but uniform distribution. Uses some older depreciated class interfaces (for mapper and reducer) This was tested on 0.20 and 0.22 (locally) so it should be fairly backwards compatible.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1354">MAPREDUCE-1354</a> | <i>Critical</i> | <b>Incremental enhancements to the JobTracker for better scalability</b></li>
|
|
</ul>
|
|
<p>Incremental enhancements to the JobTracker include a no-lock version of JT.getTaskCompletion events, no lock on the JT while doing i/o during job-submission and several fixes to cut down configuration parsing during heartbeat-handling.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1683">MAPREDUCE-1683</a> | <i>Major</i> | <b>Remove JNI calls from ClusterStatus cstr</b></li>
|
|
</ul>
|
|
<p>Removes JNI calls to get jvm current/max heap usage in ClusterStatus. Any instances of ClusterStatus serialized in a prior version will not be correctly deserialized using the updated class.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1773">MAPREDUCE-1773</a> | <i>Major</i> | <b>streaming doesn’t support jobclient.output.filter</b></li>
|
|
</ul>
|
|
<p>Improved console messaging for streaming jobs by using the generic JobClient API itself instead of the existing streaming-specific code.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1785">MAPREDUCE-1785</a> | <i>Minor</i> | <b>Add streaming config option for not emitting the key</b></li>
|
|
</ul>
|
|
<p>Added a configuration property “stream.map.input.ignoreKey” to specify whether to ignore key or not while writing input for the mapper. This configuration parameter is valid only if stream.map.input.writer.class is org.apache.hadoop.streaming.io.TextInputWriter.class. For all other InputWriter’s, key is always written.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-572">MAPREDUCE-572</a> | <i>Minor</i> | <b>If #link is missing from uri format of -cacheArchive then streaming does not throw error.</b></li>
|
|
</ul>
|
|
<p>Improved streaming job failure when #link is missing from uri format of -cacheArchive. Earlier it used to fail when launching individual tasks, now it fails during job submission itself.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1096">HDFS-1096</a> | <i>Major</i> | <b>allow dfsadmin/mradmin refresh of superuser proxy group mappings</b></li>
|
|
</ul>
|
|
<p>changed protocol name (may be used in hadoop-policy.xml) from security.refresh.usertogroups.mappings.protocol.acl to security.refresh.user.mappings.protocol.acl</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6787">HADOOP-6787</a> | <i>Major</i> | <b>Factor out glob pattern code from FileContext and Filesystem</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1836">MAPREDUCE-1836</a> | <i>Major</i> | <b>Refresh for proxy superuser config (mr part for HDFS-1096)</b></li>
|
|
</ul>
|
|
<p>changing name of the protocol (may be used in hadoop-policy.xml) from security.refresh.usertogroups.mappings.protocol.acl to security.refresh.user.mappings.protocol.acl</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1505">MAPREDUCE-1505</a> | <i>Major</i> | <b>Cluster class should create the rpc client only when needed</b></li>
|
|
</ul>
|
|
<p>Lazily construct a connection to the JobTracker from the job-submission client.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1543">MAPREDUCE-1543</a> | <i>Major</i> | <b>Log messages of JobACLsManager should use security logging of HADOOP-6586</b></li>
|
|
</ul>
|
|
<p>Adds the audit logging facility to MapReduce. All authorization/authentication events are logged to audit log. Audit log entries are stored as key=value.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1533">MAPREDUCE-1533</a> | <i>Major</i> | <b>Reduce or remove usage of String.format() usage in CapacityTaskScheduler.updateQSIObjects and Counters.makeEscapedString()</b></li>
|
|
</ul>
|
|
<p>Incremental enhancements to the JobTracker to optimize heartbeat handling.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1080">HDFS-1080</a> | <i>Major</i> | <b>SecondaryNameNode image transfer should use the defined http address rather than local ip address</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1813">MAPREDUCE-1813</a> | <i>Major</i> | <b>NPE in PipeMapred.MRErrorThread</b></li>
|
|
</ul>
|
|
<p>Fixed an NPE in streaming that occurs when there is no input to reduce and the streaming reducer sends status updates by writing “reporter:status: xxx” statements to stderr.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1829">MAPREDUCE-1829</a> | <i>Major</i> | <b>JobInProgress.findSpeculativeTask should use min() to find the candidate instead of sort()</b></li>
|
|
</ul>
|
|
<p>Improved performance of the method JobInProgress.findSpeculativeTask() which is in the critical heartbeat code path.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1887">MAPREDUCE-1887</a> | <i>Major</i> | <b>MRAsyncDiskService does not properly absolutize volume root paths</b></li>
|
|
</ul>
|
|
<p>MAPREDUCE-1887. MRAsyncDiskService now properly absolutizes volume root paths. (Aaron Kimball via zshao)</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6835">HADOOP-6835</a> | <i>Major</i> | <b>Support concatenated gzip files</b></li>
|
|
</ul>
|
|
<p>Processing of concatenated gzip files formerly stopped (quietly) at the end of the first substream/“member”; now processing will continue to the end of the concatenated stream, like gzip(1) does. (bzip2 support is unaffected by this patch.)</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1733">MAPREDUCE-1733</a> | <i>Major</i> | <b>Authentication between pipes processes and java counterparts.</b></li>
|
|
</ul>
|
|
<p>This jira introduces backward incompatibility. Existing pipes applications MUST be recompiled with new hadoop pipes library once the changes in this jira are deployed.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1315">HDFS-1315</a> | <i>Major</i> | <b>Add fsck event to audit log and remove other audit log events corresponding to FSCK listStatus and open calls</b></li>
|
|
</ul>
|
|
<p>When running fsck, audit log events are not logged for listStatus and open are not logged. A new event with cmd=fsck is logged with ugi field set to the user requesting fsck and src field set to the fsck path.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1866">MAPREDUCE-1866</a> | <i>Minor</i> | <b>Remove deprecated class org.apache.hadoop.streaming.UTF8ByteArrayUtils</b></li>
|
|
</ul>
|
|
<p>Removed public deprecated class org.apache.hadoop.streaming.UTF8ByteArrayUtils.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-330">HDFS-330</a> | <i>Trivial</i> | <b>Datanode Web UIs should provide robots.txt</b></li>
|
|
</ul>
|
|
<p>A robots.txt is now in place which will prevent well behaved crawlers from perusing Hadoop web interfaces.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-202">HDFS-202</a> | <i>Major</i> | <b>Add a bulk FIleSystem.getFileBlockLocations</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1780">MAPREDUCE-1780</a> | <i>Major</i> | <b>AccessControlList.toString() is used for serialization of ACL in JobStatus.java</b></li>
|
|
</ul>
|
|
<p>Fixes serialization of job-acls in JobStatus to use AccessControlList.write() instead of AccessControlList.toString().</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6905">HADOOP-6905</a> | <i>Major</i> | <b>Better logging messages when a delegation token is invalid</b></li>
|
|
</ul>
|
|
<p><b>WARNING: No release note provided for this change.</b></p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6693">HADOOP-6693</a> | <i>Major</i> | <b>Add metrics to track kerberos login activity</b></li>
|
|
</ul>
|
|
<p>New metrics “login” of type MetricTimeVaryingRate is added under new metrics context name “ugi” and metrics record name “ugi”.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1318">HDFS-1318</a> | <i>Major</i> | <b>HDFS Namenode and Datanode WebUI information needs to be accessible programmatically for scripts</b></li>
|
|
</ul>
|
|
<p>resubmit the patch for HDFS1318 as Hudson was down last week.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-220">MAPREDUCE-220</a> | <i>Major</i> | <b>Collecting cpu and memory usage for MapReduce tasks</b></li>
|
|
</ul>
|
|
<p>Collect cpu and memory statistics per task.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-712">HDFS-712</a> | <i>Major</i> | <b>Move libhdfs from mr to hdfs</b></li>
|
|
</ul>
|
|
<p>Moved the libhdfs package to the HDFS subproject.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2032">MAPREDUCE-2032</a> | <i>Major</i> | <b>TestJobOutputCommitter fails in ant test run</b></li>
|
|
</ul>
|
|
<p>Clears a problem that {{TestJobCleanup}} leaves behind files that cause {{TestJobOutputCommitter}} to error out.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6922">HADOOP-6922</a> | <i>Major</i> | <b>COMMON part of MAPREDUCE-1664</b></li>
|
|
</ul>
|
|
<p>Makes AccessControlList a writable and updates documentation for Job ACLs.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1664">MAPREDUCE-1664</a> | <i>Major</i> | <b>Job Acls affect Queue Acls</b></li>
|
|
</ul><!-- markdown -->
|
|
|
|
<ul>
|
|
|
|
<li>Removed aclsEnabled flag from queues configuration files.</li>
|
|
<li>Removed the configuration property mapreduce.cluster.job-authorization-enabled.</li>
|
|
<li>Added mapreduce.cluster.acls.enabled as the single configuration property in mapred-default.xml that enables the authorization checks for all job level and queue level operations.</li>
|
|
<li>To enable authorization of users to do job level and queue level operations, mapreduce.cluster.acls.enabled is to be set to true in JobTracker’s configuration and in all TaskTrackers’ configurations.</li>
|
|
<li>To get access to a job, it is enough for a user to be part of one of the access lists i.e. either job-acl or queue-admins-acl(unlike before, when, one has to be part of both the lists).</li>
|
|
<li>Queue administrators(configured via acl-administer-jobs) of a queue can do all view-job and modify-job operations on all jobs submitted to that queue.</li>
|
|
<li>ClusterOwner(who started the mapreduce cluster) and cluster administrators(configured via mapreduce.cluster.permissions.supergroup) can do all job level operations and queue level operations on all jobs on all queues in that cluster irrespective of job-acls and queue-acls configured.</li>
|
|
<li>JobOwner(who submitted job to a queue) can do all view-job and modify-job operations on his/her job irrespective of job-acls and queue-acls.</li>
|
|
<li>Since aclsEnabled flag is removed from queues configuration files, “refresh of queues configuration” will not change mapreduce.cluster.acls.enabled on the fly. mapreduce.cluster.acls.enabled can be modified only when restarting the mapreduce cluster.</li>
|
|
</ul><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1517">MAPREDUCE-1517</a> | <i>Major</i> | <b>streaming should support running on background</b></li>
|
|
</ul>
|
|
<p>Adds -background option to run a streaming job in background.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2147">MAPREDUCE-2147</a> | <i>Trivial</i> | <b>JobInProgress has some redundant lines in its ctor</b></li>
|
|
</ul>
|
|
<p>Remove some redundant lines from JobInProgress’s constructor which was re-initializing things unnecessarily.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1435">HDFS-1435</a> | <i>Major</i> | <b>Provide an option to store fsimage compressed</b></li>
|
|
</ul>
|
|
<p>This provides an option to store fsimage compressed. The layout version is bumped to -25. The user could configure if s/he wants the fsimage to be compressed or not and which codec to use. By default the fsimage is not compressed.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7005">HADOOP-7005</a> | <i>Major</i> | <b>Update test-patch.sh to remove callback to Hudson master</b></li>
|
|
</ul>
|
|
<p>N/A</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6663">HADOOP-6663</a> | <i>Major</i> | <b>BlockDecompressorStream get EOF exception when decompressing the file compressed from empty file</b></li>
|
|
</ul>
|
|
<p>Fix EOF exception in BlockDecompressorStream when decompressing previous compressed empty file</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-903">HDFS-903</a> | <i>Critical</i> | <b>NN should verify images and edit logs on startup</b></li>
|
|
</ul>
|
|
<p>Store fsimage MD5 checksum in VERSION file. Validate checksum when loading a fsimage. Layout version bumped.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1457">HDFS-1457</a> | <i>Major</i> | <b>Limit transmission rate when transfering image between primary and secondary NNs</b></li>
|
|
</ul>
|
|
<p>Add a configuration variable dfs.image.transfer.bandwidthPerSec to allow the user to specify the amount of bandwidth for transferring image and edits. Its default value is 0 indicating no throttling.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1035">HDFS-1035</a> | <i>Major</i> | <b>Generate Eclipse’s .classpath file from Ivy config</b></li>
|
|
</ul>
|
|
<p>Added support to auto-generate the Eclipse .classpath file from ivy.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1592">MAPREDUCE-1592</a> | <i>Major</i> | <b>Generate Eclipse’s .classpath file from Ivy config</b></li>
|
|
</ul>
|
|
<p>Added support to auto-generate the Eclipse .classpath file from ivy.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-4675">HADOOP-4675</a> | <i>Major</i> | <b>Current Ganglia metrics implementation is incompatible with Ganglia 3.1</b></li>
|
|
</ul>
|
|
<p>Support for reporting metrics to Ganglia 3.1 servers</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1905">MAPREDUCE-1905</a> | <i>Blocker</i> | <b>Context.setStatus() and progress() api are ignored</b></li>
|
|
</ul>
|
|
<p>Moved the api public Counter getCounter(Enum<?> counterName), public Counter getCounter(String groupName, String counterName) from org.apache.hadoop.mapreduce.TaskInputOutputContext to org.apache.hadoop.mapreduce.TaskAttemptContext</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7013">HADOOP-7013</a> | <i>Major</i> | <b>Add boolean field isCorrupt to BlockLocation</b></li>
|
|
</ul>
|
|
<p>This patch has changed the serialization format of BlockLocation.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6683">HADOOP-6683</a> | <i>Minor</i> | <b>the first optimization: ZlibCompressor does not fully utilize the buffer</b></li>
|
|
</ul>
|
|
<p>Improve the buffer utilization of ZlibCompressor to avoid invoking a JNI per write request.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1560">HDFS-1560</a> | <i>Minor</i> | <b>dfs.data.dir permissions should default to 700</b></li>
|
|
</ul>
|
|
<p>The permissions on datanode data directories (configured by dfs.datanode.data.dir.perm) now default to 0700. Upon startup, the datanode will automatically change the permissions to match the configured value.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2096">MAPREDUCE-2096</a> | <i>Blocker</i> | <b>Secure local filesystem IO from symlink vulnerabilities</b></li>
|
|
</ul>
|
|
<p>The TaskTracker now uses the libhadoop JNI library to operate securely on local files when security is enabled. Secure clusters must ensure that libhadoop.so is available to the TaskTracker.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7089">HADOOP-7089</a> | <i>Minor</i> | <b>Fix link resolution logic in hadoop-config.sh</b></li>
|
|
</ul>
|
|
<p>Updates hadoop-config.sh to always resolve symlinks when determining HADOOP_HOME. Bash built-ins or POSIX:2001 compliant cmds are now required.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6436">HADOOP-6436</a> | <i>Major</i> | <b>Remove auto-generated native build files</b></li>
|
|
</ul>
|
|
<p>The native build run when from trunk now requires autotools, libtool and openssl dev libraries.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2260">MAPREDUCE-2260</a> | <i>Major</i> | <b>Remove auto-generated native build files</b></li>
|
|
</ul>
|
|
<p>The native build run when from trunk now requires autotools, libtool and openssl dev libraries.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1582">HDFS-1582</a> | <i>Major</i> | <b>Remove auto-generated native build files</b></li>
|
|
</ul>
|
|
<p>The native build run when from trunk now requires autotools, libtool and openssl dev libraries.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7134">HADOOP-7134</a> | <i>Major</i> | <b>configure files that are generated as part of the released tarball need to have executable bit set</b></li>
|
|
</ul>
|
|
<p>I have just committed this to trunk and branch-0.22. Thanks Roman!</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2054">MAPREDUCE-2054</a> | <i>Major</i> | <b>Hierarchical queue implementation broke dynamic queue addition in Dynamic Scheduler</b></li>
|
|
</ul>
|
|
<p>Fix Dynamic Priority Scheduler to work with hierarchical queue names</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1996">MAPREDUCE-1996</a> | <i>Trivial</i> | <b>API: Reducer.reduce() method detail misstatement</b></li>
|
|
</ul>
|
|
<p>Fix a misleading documentation note about the usage of Reporter objects in Reducers.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1159">MAPREDUCE-1159</a> | <i>Trivial</i> | <b>Limit Job name on jobtracker.jsp to be 80 char long</b></li>
|
|
</ul>
|
|
<p>Job names on jobtracker.jsp should be 80 characters long at most.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2272">MAPREDUCE-2272</a> | <i>Trivial</i> | <b>Job ACL file should not be executable</b></li>
|
|
</ul>
|
|
<p>Job ACL files now have permissions set to 600 (previously 700).</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2251">MAPREDUCE-2251</a> | <i>Major</i> | <b>Remove mapreduce.job.userhistorylocation config</b></li>
|
|
</ul>
|
|
<p>Remove the now defunct property `mapreduce.job.userhistorylocation`.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7156">HADOOP-7156</a> | <i>Critical</i> | <b>getpwuid_r is not thread-safe on RHEL6</b></li>
|
|
</ul>
|
|
<p>Adds a new configuration hadoop.work.around.non.threadsafe.getpwuid which can be used to enable a mutex around this call to workaround thread-unsafe implementations of getpwuid_r. Users should consult <a class="externalLink" href="http://wiki.apache.org/hadoop/KnownBrokenPwuidImplementations">http://wiki.apache.org/hadoop/KnownBrokenPwuidImplementations</a> for a list of such systems.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1596">HDFS-1596</a> | <i>Major</i> | <b>Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml</b></li>
|
|
</ul>
|
|
<p>Removed references to the older fs.checkpoint.* properties that resided in core-site.xml</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7117">HADOOP-7117</a> | <i>Major</i> | <b>Move secondary namenode checkpoint configs from core-default.xml to hdfs-default.xml</b></li>
|
|
</ul>
|
|
<p>Removed references to the older fs.checkpoint.* properties that resided in core-site.xml</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-6949">HADOOP-6949</a> | <i>Major</i> | <b>Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting</b></li>
|
|
</ul>
|
|
<p>Increments the RPC protocol version in org.apache.hadoop.ipc.Server from 4 to 5. Introduces ArrayPrimitiveWritable for a much more efficient wire format to transmit arrays of primitives over RPC. ObjectWritable uses the new writable for array of primitives for RPC and continues to use existing format for on-disk data.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7193">HADOOP-7193</a> | <i>Minor</i> | <b>Help message is wrong for touchz command.</b></li>
|
|
</ul>
|
|
<p>Updated the help for the touchz command.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7229">HADOOP-7229</a> | <i>Major</i> | <b>Absolute path to kinit in auto-renewal thread</b></li>
|
|
</ul>
|
|
<p>When Hadoop’s Kerberos integration is enabled, it is now required that either {{kinit}} be on the path for user accounts running the Hadoop client, or that the {{hadoop.kerberos.kinit.command}} configuration option be manually set to the absolute path to {{kinit}}.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2410">MAPREDUCE-2410</a> | <i>Minor</i> | <b>document multiple keys per reducer oddity in hadoop streaming FAQ</b></li>
|
|
</ul>
|
|
<p>Add an FAQ entry regarding the differences between Java API and Streaming development of MR programs.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1825">HDFS-1825</a> | <i>Major</i> | <b>Remove thriftfs contrib</b></li>
|
|
</ul>
|
|
<p>Removed thriftfs contrib component.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7137">HADOOP-7137</a> | <i>Major</i> | <b>Remove hod contrib</b></li>
|
|
</ul>
|
|
<p>Removed contrib related build targets.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7192">HADOOP-7192</a> | <i>Trivial</i> | <b>fs -stat docs aren’t updated to reflect the format features</b></li>
|
|
</ul>
|
|
<p>Updated the web documentation to reflect the formatting abilities of ‘fs -stat’.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HADOOP-7302">HADOOP-7302</a> | <i>Major</i> | <b>webinterface.private.actions should not be in common</b></li>
|
|
</ul>
|
|
<p>Option webinterface.private.actions has been renamed to mapreduce.jobtracker.webinterface.trusted and should be specified in mapred-site.xml instead of core-site.xml</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-2516">MAPREDUCE-2516</a> | <i>Minor</i> | <b>option to control sensitive web actions</b></li>
|
|
</ul>
|
|
<p>Configuration option webinterface.private.actions has been renamed to mapreduce.jobtracker.webinterface.trusted</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/HDFS-1948">HDFS-1948</a> | <i>Major</i> | <b>Forward port ‘hdfs-1520 lightweight namenode operation to trigger lease reccovery’</b></li>
|
|
</ul>
|
|
<p>Adds method to NameNode/ClientProtocol that allows for rude revoke of lease on current lease holder</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-3151">MAPREDUCE-3151</a> | <i>Major</i> | <b>Contrib tests failing</b></li>
|
|
</ul>
|
|
<p>Confirmed that problem of finding ivy file occurs w/o patch with ant 1.7, and not with patch (with either ant 1.7 or 1.8). Other unit tests are still failing the test steps themselves on my laptop, but that is not due not finding the ivy file.</p><hr />
|
|
<ul>
|
|
|
|
<li><a class="externalLink" href="https://issues.apache.org/jira/browse/MAPREDUCE-1118">MAPREDUCE-1118</a> | <i>Major</i> | <b>Capacity Scheduler scheduling information is hard to read / should be tabular format</b></li>
|
|
</ul>
|
|
<p>Add CapacityScheduler servlet to enhance web UI for queue information.</p>
|
|
</div>
|
|
</div>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
<div id="footer">
|
|
<div class="xright">
|
|
© 2008-2023
|
|
Apache Software Foundation
|
|
|
|
- <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a>.
|
|
Apache Maven, Maven, Apache, the Apache feather logo, and the Apache Maven project logos are trademarks of The Apache Software Foundation.
|
|
</div>
|
|
<div class="clear">
|
|
<hr/>
|
|
</div>
|
|
</div>
|
|
</body>
|
|
</html>
|