Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Updating all CHANGES.txt files to move entires from future releases into 2.6.1 

section given the large number of backports to 2.6.1.

(cherry picked from commit fbbb7ff1ed)
This commit is contained in:
Vinod Kumar Vavilapalli (I am also known as @tshooter.) 2015-09-10 13:45:57 -07:00
parent 5decd8f9f7
commit cbf90f93f9
4 changed files with 408 additions and 413 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

112
hadoop-common-project/hadoop-common/CHANGES.txt
View File

@ -243,9 +243,6 @@ Release 2.8.0 - UNRELEASED
command-line arguments passed by the user (Masatake Iwasaki via Colin P.
McCabe)
HADOOP-12280. Skip unit tests based on maven profile rather than
NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe)
HADOOP-12318. Expose underlying LDAP exceptions in SaslPlainServer. (Mike
Yoder via atm)
@ -266,9 +263,6 @@ Release 2.8.0 - UNRELEASED
HADOOP-11785. Reduce the number of listStatus operation in distcp
buildListing (Zoran Dimitrijevic via Colin P. McCabe)
HADOOP-11812. Implement listLocatedStatus for ViewFileSystem to speed up
split calculation (gera)
HADOOP-11970. Replace uses of ThreadLocal<Random> with JDK7
ThreadLocalRandom. (Sean Busbey via Colin P. McCabe)
@ -304,9 +298,6 @@ Release 2.8.0 - UNRELEASED
HADOOP-12352. Delay in checkpointing Trash can leave trash for 2 intervals
before deleting (Casey Brotherton via harsh)
HADOOP-11802: DomainSocketWatcher thread terminates sometimes after there
is an I/O error during requestShortCircuitShm (cmccabe)
HADOOP-11568. Description on usage of classpath in hadoop command is
incomplete. ( Archana T via vinayakumarb )
@ -358,9 +349,6 @@ Release 2.8.0 - UNRELEASED
HADOOP-11866. increase readability and reliability of checkstyle,
shellcheck, and whitespace reports (aw)
HADOOP-11491. HarFs incorrectly declared as requiring an authority.
(Brahma Reddy Battula via gera)
HADOOP-11889. Make checkstyle runnable from root project
(Gera Shegalov via jeagles)
@ -632,9 +620,6 @@ Release 2.7.2 - UNRELEASED
HADOOP-12304. Applications using FileContext fail with the default file
system configured to be wasb/s3/etc. (cnauroth)
HADOOP-11932. MetricsSinkAdapter may hang when being stopped.
(Brahma Reddy Battula via jianhe)
HADOOP-12061. Incorrect command in single cluster setup document.
(Kengo Seki via aajisaka)
@ -668,12 +653,6 @@ Release 2.7.1 - 2015-07-06
HADOOP-11872. "hadoop dfs" command prints message about using "yarn jar" on
Windows(branch-2 only) (Varun Vasudev via cnauroth)
HADOOP-11730. Regression: s3n read failure recovery broken.
(Takenori Sato via stevel)
HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there
is an I/O error during requestShortCircuitShm (cmccabe)
HADOOP-11891. OsSecureRandom should lazily fill its reservoir (asuresh)
HADOOP-9658. SnappyCodec#checkNativeCodeLoaded may unexpectedly fail when
@ -688,9 +667,6 @@ Release 2.7.1 - 2015-07-06
HADOOP-11973. Ensure ZkDelegationTokenSecretManager namespace znodes get
created with ACLs. (Gregory Chanan via asuresh)
HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes
infinite loop. (Larry McCay via cnauroth)
HADOOP-12058. Fix dead links to DistCp and Hadoop Archives pages.
(Kazuho Fujii via aajisaka)
@ -957,18 +933,12 @@ Release 2.7.0 - 2015-04-20
HADOOP-11323. WritableComparator#compare keeps reference to byte array.
(Wilfred Spiegelenburg via wang)
HADOOP-11238. Update the NameNode's Group Cache in the background when
possible (Chris Li via Colin P. McCabe)
HADOOP-10809. hadoop-azure: page blob support. (Dexter Bradshaw,
Mostafa Elhemali, Eric Hanson, and Mike Liddell via cnauroth)
HADOOP-11188. hadoop-azure: automatically expand page blobs when they become
full. (Eric Hanson via cnauroth)
HADOOP-11506. Configuration variable expansion regex expensive for long
values. (Gera Shegalov via gera)
HADOOP-11231. Remove dead code in ServletUtil. (Li Lu via wheat9)
HADOOP-11620. Add support for load balancing across a group of KMS for HA.
@ -1083,9 +1053,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11355. When accessing data in HDFS and the key has been deleted,
a Null Pointer Exception is shown. (Arun Suresh via wang)
HADOOP-11343. Overflow is not properly handled in caclulating final iv for
AES CTR. (Jerry Chen via wang)
HADOOP-11354. ThrottledInputStream doesn't perform effective throttling.
(Ted Yu via jing9)
@ -1101,9 +1068,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11369. Fix new findbugs warnings in hadoop-mapreduce-client,
non-core directories. (Li Lu via wheat9)
HADOOP-11368. Fix SSLFactory truststore reloader thread leak in
KMSClientProvider. (Arun Suresh via wang)
HADOOP-11372. Fix new findbugs warnings in mapreduce-examples.
(Li Lu via wheat9)
@ -1201,9 +1165,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11462. TestSocketIOWithTimeout needs change for PowerPC platform.
(Ayappan via cnauroth)
HADOOP-11350. The size of header buffer of HttpServer is too small when
HTTPS is enabled. (Benoy Antony via wheat9)
HADOOP-10542 Potential null pointer dereference in Jets3tFileSystemStore
retrieveBlock(). (Ted Yu via stevel)
@ -1228,9 +1189,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11507 Hadoop RPC Authentication problem with different user locale.
(Talat UYARER via stevel)
HADOOP-11482. Use correct UGI when KMSClientProvider is called by a proxy
user. Contributed by Arun Suresh.
HADOOP-11499. Check of executorThreadsStarted in
ValueQueue#submitRefillTask() evades lock acquisition (Ted Yu via jlowe)
@ -1302,9 +1260,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11570. S3AInputStream.close() downloads the remaining bytes of
the object from S3. (Dan Hecht via stevel).
HADOOP-11295. RPC Server Reader thread can't shutdown if RPCCallQueue is
full. (Ming Ma via kihwal)
HADOOP-11599. Client#getTimeout should use IPC_CLIENT_PING_DEFAULT when
IPC_CLIENT_PING_KEY is not configured. (zhihai xu via ozawa)
@ -1317,9 +1272,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-9087. Queue size metric for metric sinks isn't actually maintained
(Akira AJISAKA via jlowe)
HADOOP-11604. Prevent ConcurrentModificationException while closing domain
sockets during shutdown of DomainSocketWatcher thread. (cnauroth)
HADOOP-11612. Workaround for Curator's ChildReaper requiring Guava 15+.
(rkanter)
@ -1356,9 +1308,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11666. Revert the format change of du output introduced by
HADOOP-6857. (Byron Wong via aajisaka)
HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream
should be non static. (Sean Busbey via yliu)
HADOOP-11670. Regression: s3a auth setup broken. (Adam Budde via stevel)
HADOOP-11602. Fix toUpperCase/toLowerCase to use Locale.ENGLISH. (ozawa)
@ -1372,9 +1321,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11693. Azure Storage FileSystem rename operations are throttled too
aggressively to complete HBase WAL archiving. (Duo Xu via cnauroth)
HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt
synchronization. (Sean Busbey via yliu)
HADOOP-11558. Fix dead links to doc of hadoop-tools. (Jean-Pierre
Matsumoto via ozawa)
@ -1450,7 +1396,7 @@ Release 2.6.2 - UNRELEASED
BUG FIXES
Release 2.6.1 - UNRELEASED
Release 2.6.1 - 2015-09-09
INCOMPATIBLE CHANGES
@ -1461,11 +1407,20 @@ Release 2.6.1 - UNRELEASED
HADOOP-7139. Allow appending to existing SequenceFiles
(kanaka kumar avvaru via vinayakumarb)
HADOOP-12280. Skip unit tests based on maven profile rather than
NativeCodeLoader.isNativeCodeLoaded (Masatake Iwasaki via Colin P. McCabe)
OPTIMIZATIONS
HADOOP-11238. Update the NameNode's Group Cache in the background when
possible (Chris Li via Colin P. McCabe)
HADOOP-11506. Configuration variable expansion regex expensive for long
values. (Gera Shegalov via gera)
BUG FIXES
HADOOP-11466. FastByteComparisons: do not use UNSAFE_COMPARER on the SPARC
HADOOP-11466: FastByteComparisons: do not use UNSAFE_COMPARER on the SPARC
architecture because it is slower there (Suman Somasundar via Colin P.
McCabe)
@ -1474,6 +1429,51 @@ Release 2.6.1 - UNRELEASED
HADOOP-11333. Fix deadlock in DomainSocketWatcher when the notification
pipe is full (zhaoyunjiong via cmccabe)
HADOOP-11343. Overflow is not properly handled in caclulating final iv for
AES CTR. (Jerry Chen via wang)
HADOOP-11368. Fix SSLFactory truststore reloader thread leak in
KMSClientProvider. (Arun Suresh via wang)
HADOOP-11350. The size of header buffer of HttpServer is too small when
HTTPS is enabled. (Benoy Antony via wheat9)
HADOOP-11482. Use correct UGI when KMSClientProvider is called by a proxy
user. Contributed by Arun Suresh.
HADOOP-11295. RPC Server Reader thread can't shutdown if RPCCallQueue is
full. (Ming Ma via kihwal)
HADOOP-11604. Prevent ConcurrentModificationException while closing domain
sockets during shutdown of DomainSocketWatcher thread. (cnauroth)
HADOOP-11674. oneByteBuf in CryptoInputStream and CryptoOutputStream
should be non static. (Sean Busbey via yliu)
HADOOP-11710. Make CryptoOutputStream behave like DFSOutputStream wrt
synchronization. (Sean Busbey via yliu)
HADOOP-11812. Implement listLocatedStatus for ViewFileSystem to speed up
split calculation (gera)
HADOOP-11730. Regression: s3n read failure recovery broken.
(Takenori Sato via stevel)
HADOOP-11802: DomainSocketWatcher thread terminates sometimes after there
is an I/O error during requestShortCircuitShm (cmccabe)
HADOOP-11491. HarFs incorrectly declared as requiring an authority.
(Brahma Reddy Battula via gera)
HADOOP-8151. Error handling in snappy decompressor throws invalid
exceptions. (Matt Foley via harsh)
HADOOP-11932. MetricsSinkAdapter may hang when being stopped.
(Brahma Reddy Battula via jianhe)
HADOOP-11934. Use of JavaKeyStoreProvider in LdapGroupsMapping causes
infinite loop. (Larry McCay via cnauroth)
Release 2.6.0 - 2014-11-18
INCOMPATIBLE CHANGES

263
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
View File

@ -93,8 +93,6 @@ Release 2.8.0 - UNRELEASED
HDFS-8085. Move CorruptFileBlockIterator to a new hdfs.client.impl package.
(szetszwo)
HDFS-8046. Allow better control of getContentSummary (kihwal)
HDFS-8076. Code cleanup for DFSInputStream: use offset instead of
LocatedBlock when possible. (Zhe Zhang via wang)
@ -495,9 +493,6 @@ Release 2.8.0 - UNRELEASED
HDFS-8900. Compact XAttrs to optimize memory footprint. (yliu)
HDFS-8846. Add a unit test for INotify functionality across a layout
version upgrade (Zhe Zhang via Colin P. McCabe)
HDFS-8951. Move the shortcircuit package to hdfs-client.
(Mingliang Liu via wheat9)
@ -546,9 +541,6 @@ Release 2.8.0 - UNRELEASED
HDFS-9012. Move o.a.h.hdfs.protocol.datatransfer.PipelineAck class to
hadoop-hdfs-client module. (Mingliang Liu via wheat9)
HDFS-8384. Allow NN to startup if there are files having a lease but are not
under construction. (jing9)
HDFS-8984. Move replication queues related methods in FSNamesystem to
BlockManager. (wheat9)
@ -696,9 +688,6 @@ Release 2.8.0 - UNRELEASED
HDFS-8290. WebHDFS calls before namesystem initialization can cause
NullPointerException. (cnauroth)
HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart.
(surendra singh lilhore via Xiaoyu Yao)
HDFS-8310. Fix TestCLI.testAll "help: help for find" on Windows.
(Kiran Kumar M R via Xiaoyu Yao)
@ -805,18 +794,12 @@ Release 2.8.0 - UNRELEASED
HDFS-8268. Port conflict log for data node server is not sufficient
(Mohammad Shahid Khan via vinayakumarb)
HDFS-8431. hdfs crypto class not found in Windows.
(Anu Engineer via cnauroth)
HDFS-8407. hdfsListDirectory must set errno to 0 on success (Masatake
Iwasaki via Colin P. McCabe)
HDFS-7401. Add block info to DFSInputStream' WARN message when it adds
node to deadNodes (Arshad Mohammad via vinayakumarb)
HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits.
(Ming Ma via jing9)
HDFS-8490. Typo in trace enabled log in ExceptionHandler of WebHDFS.
(Archana T via ozawa)
@ -1018,9 +1001,6 @@ Release 2.7.2 - UNRELEASED
HDFS-8867. Enable optimized block reports. (Daryn Sharp via jing9)
HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is
flawed. (Kihwal Lee via yliu)
HDFS-8891. HDFS concat should keep srcs order. (Yong Zhang via jing9)
HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect.
@ -1065,15 +1045,9 @@ Release 2.7.1 - 2015-07-06
(Surendra Singh Lilhore via szetszwo)
OPTIMIZATIONS
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using
hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang
via Colin P. McCabe)
BUG FIXES
HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to
finalize upgrade. (jing9)
HDFS-8151. Always use snapshot path as source when invalid snapshot names
are used for diff based distcp. (jing9)
@ -1095,9 +1069,6 @@ Release 2.7.1 - 2015-07-06
HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode.
(surendra singh lilhore via szetszwo)
HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on
post-HDFS-7915 DataNode (cmccabe)
HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the
lock. (wheat9)
@ -1110,17 +1081,8 @@ Release 2.7.1 - 2015-07-06
HDFS-8305: HDFS INotify: the destination field of RenameOp should always
end with the file name (cmccabe)
HDFS-7980. Incremental BlockReport will dramatically slow down namenode
startup. (Walter Su via szetszwo)
HDFS-8226. Non-HA rollback compatibility broken (J.Andreina via vinayakumarb)
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query
command is issued. (Brahma Reddy Battula via kihwal)
HDFS-8254. Standby namenode doesn't process DELETED_BLOCK if the add block
request is in edit log. (Rushabh S Shah via kihwal)
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor
goes for infinite loop (Rushabh S Shah via kihwal)
@ -1129,9 +1091,6 @@ Release 2.7.1 - 2015-07-06
HDFS-8405. Fix a typo in NamenodeFsck. (Takanobu Asanuma via szetszwo)
HDFS-8404. Pending block replication can get stuck using older genstamp
(Nathan Roberts via kihwal)
HDFS-8451. DFSClient probe for encryption testing interprets empty URI
property for "enabled". (Steve Loughran via xyao)
@ -1183,9 +1142,6 @@ Release 2.7.0 - 2015-04-20
NEW FEATURES
HDFS-7278. Add a command that allows sysadmins to manually trigger full
block reports from a DN (cmccabe)
HDFS-6663. Admin command to track file and locations from block id.
(Chen He via kihwal)
@ -1279,9 +1235,6 @@ Release 2.7.0 - 2015-04-20
HDFS-3342. SocketTimeoutException in BlockSender.sendChunks could
have a better error message. (Yongjun Zhang via wang)
HDFS-7035. Make adding a new data directory to the DataNode an atomic
operation and improve error handling (Lei Xu via Colin P. McCabe)
HDFS-6917. Add an hdfs debug command to validate blocks, call recoverlease,
etc. (cmccabe)
@ -1367,9 +1320,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7462. Consolidate implementation of mkdirs() into a single class.
(wheat9)
HDFS-7446. HDFS inotify should have the ability to determine what txid it
has read up to (cmccabe)
HDFS-6735. A minor optimization to avoid pread() be blocked by read()
inside the same DFSInputStream (Lars Hofhansl via stack)
@ -1412,9 +1362,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7528. Consolidate symlink-related implementation into a single class.
(wheat9)
HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin
P. McCabe)
HDFS-7373. Clean up temporary files after fsimage transfer failures.
(kihwal)
@ -1432,8 +1379,6 @@ Release 2.7.0 - 2015-04-20
HADOOP-11470. Remove some uses of obsolete guava APIs from the hadoop
codebase. (Sangjin Lee via Colin P. McCabe)
HDFS-7182. JMX metrics aren't accessible when NN is busy. (Ming Ma via jing9)
HDFS-7323. Move the get/setStoragePolicy commands out from dfsadmin.
(jing9 via yliu)
@ -1633,9 +1578,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7410. Support CreateFlags with append() to support hsync() for
appending streams (Vinayakumar B via Colin P. McCabe)
HDFS-7742. Favoring decommissioning node for replication can cause a block
to stay underreplicated for long periods (Nathan Roberts via kihwal)
HDFS-8008. Support client-side back off when the datanodes are congested.
(wheat9)
@ -1786,9 +1728,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7431. log message for InvalidMagicNumberException may be incorrect.
(Yi Liu via cnauroth)
HDFS-7552. Change FsVolumeList toString() to fix
TestDataNodeVolumeFailureToleration (Liang Xie via Colin P. McCabe)
HDFS-7557. Fix spacing for a few keys in DFSConfigKeys.java
(Colin P.McCabe)
@ -1815,21 +1754,9 @@ Release 2.7.0 - 2015-04-20
HDFS-7589. Break the dependency between libnative_mini_dfs and libhdfs.
(Zhanwei Wang via cnauroth)
HDFS-7579. Improve log reporting during block report rpc failure.
(Charles Lamb via cnauroth)
HDFS-7596. NameNode should prune dead storages from storageMap.
(Arpit Agarwal via cnauroth)
HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade
shutdown command (Eric Payne via kihwal)
HDFS-5445. PacketReceiver populates the packetLen field in PacketHeader
incorrectly (Jonathan Mace via Colin P. McCabe)
HDFS-7470. SecondaryNameNode need twice memory when calling
reloadFromImageFile. (zhaoyunjiong via cnauroth)
HDFS-7585. Get TestEnhancedByteBufferAccess working on CPU architectures
with page sizes other than 4096 (Sam Liu via Colin P. McCabe)
@ -1847,15 +1774,9 @@ Release 2.7.0 - 2015-04-20
HDFS-7496. Fix FsVolume removal race conditions on the DataNode by
reference-counting the volume instances (lei via cmccabe)
HDFS-7610. Fix removal of dynamically added DN volumes (Lei (Eddy) Xu via
Colin P. McCabe)
HDFS-7548. Corrupt block reporting delayed until datablock scanner thread
detects it (Rushabh Shah via kihwal)
HDFS-7575. Upgrade should generate a unique storage ID for each
volume. (Arpit Agarwal)
HDFS-3519. Checkpoint upload may interfere with a concurrent saveNamespace.
(Ming Ma via cnauroth)
@ -1896,9 +1817,6 @@ Release 2.7.0 - 2015-04-20
HDFS-6651. Deletion failure can leak inodes permanently.
(Jing Zhao via wheat9)
HDFS-7707. Edit log corruption due to delayed block removal again.
(Yongjun Zhang via kihwal)
HDFS-7734. Class cast exception in NameNode#main. (yliu via wang)
HDFS-7719. BlockPoolSliceStorage#removeVolumes fails to remove some
@ -1925,9 +1843,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7718. Store KeyProvider in ClientContext to avoid leaking key provider
threads when using FileContext (Arun Suresh via Colin P. McCabe)
HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause
DataNode to register successfully with only one NameNode.(vinayakumarb)
HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir.
(szetszwo)
@ -1964,15 +1879,9 @@ Release 2.7.0 - 2015-04-20
HDFS-6662. WebHDFS cannot open a file if its path contains "%".
(Gerson Carlos via wheat9)
HDFS-7788. Post-2.6 namenode may not start up with an image containing
inodes created with an old release. (Rushabh Shah via kihwal)
HDFS-7814. Fix usage string of storageType parameter for
"dfsadmin -setSpaceQuota/clrSpaceQuota". (Xiaoyu Yao via cnauroth)
HDFS-7009. Active NN and standby NN have different live nodes.
(Ming Ma via cnauroth)
HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe)
HDFS-7805. NameNode recovery prompt should be printed on console (Surendra
@ -1984,9 +1893,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7831. Fix the starting index and end condition of the loop in
FileDiffList.findEarlierSnapshotBlocks(). (Konstantin Shvachko via jing9)
HDFS-7763. fix zkfc hung issue due to not catching exception in a corner
case. (Liang Xie via wang)
HDFS-7843. A truncated file is corrupted after rollback from a rolling
upgrade. (szetszwo)
@ -1999,9 +1905,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7785. Improve diagnostics information for HttpPutFailedException.
(Chengbing Liu via wheat9)
HDFS-7871. NameNodeEditLogRoller can keep printing "Swallowing exception"
message. (jing9)
HDFS-7757. Misleading error messages in FSImage.java. (Brahma Reddy Battula
via Arpit Agarwal)
@ -2019,15 +1922,9 @@ Release 2.7.0 - 2015-04-20
HDFS-7434. DatanodeID hashCode should not be mutable. (daryn via kihwal)
HDFS-7885. Datanode should not trust the generation stamp provided by
client. (Tsz Wo Nicholas Sze via jing9)
HDFS-7818. OffsetParam should return the default value instead of throwing
NPE when the value is unspecified. (Eric Payne via wheat9)
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. Mccabe)
HDFS-7880. Remove the tests for legacy Web UI in branch-2.
(Brahma Reddy Battula via aajisaka)
@ -2040,9 +1937,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7903. Cannot recover block after truncate and delete snapshot.
(Plamen Jeliazkov via shv)
HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and
fail to tell the DFSClient about it because of a network error (cmccabe)
HDFS-7886. Fix TestFileTruncate falures. (Plamen Jeliazkov and shv)
HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Xiaoyu Yao
@ -2076,18 +1970,10 @@ Release 2.7.0 - 2015-04-20
HDFS-7943. Append cannot handle the last block with length greater than
the preferred block size. (jing9)
HDFS-7587. Edit log corruption can happen if append fails with a quota
violation. (jing9)
HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade
starts (Zhe Zhang via Colin P. McCabe)
HDFS-7816. Unable to open webhdfs paths with "+". (wheat9 via kihwal)
HDFS-7932. Speed up the shutdown of datanode during rolling upgrade.(kihwal)
HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu)
HDFS-7957. Truncate should verify quota before making changes. (jing9)
HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now()
@ -2095,16 +1981,6 @@ Release 2.7.0 - 2015-04-20
HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts (brandonli)
HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2.
(Brahma Reddy Battula via aajisaka)
HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp
provided by the client is larger than the one stored in the datanode.
(Brahma Reddy Battula via szetszwo)
HDFS-7960. The full block report should prune zombie storages even if
they're not empty. (cmccabe and Eddy Xu via wang)
HDFS-7961. Trigger full block report after hot swapping disk. (Eddy Xu via wang)
HDFS-7977. NFS couldn't take percentile intervals (brandonli)
@ -2137,15 +2013,9 @@ Release 2.7.0 - 2015-04-20
HDFS-8051. FsVolumeList#addVolume should release volume reference if not
put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe)
HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl
lock for a very long time (sinago via cmccabe)
HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in
platform-specific format. (Xiaoyu Yao via cnauroth)
HDFS-8072. Reserved RBW space is not released if client terminates while
writing block. (Arpit Agarwal)
HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can
complete without blocks being replicated. (Ming Ma via wang)
@ -2190,16 +2060,42 @@ Release 2.6.2 - UNRELEASED
BUG FIXES
Release 2.6.1 - UNRELEASED
Release 2.6.1 - 2015-09-09
INCOMPATIBLE CHANGES
NEW FEATURES
HDFS-7278. Add a command that allows sysadmins to manually trigger full
block reports from a DN (cmccabe)
IMPROVEMENTS
HDFS-7035. Make adding a new data directory to the DataNode an atomic
operation and improve error handling (Lei Xu via Colin P. McCabe)
HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin
P. McCabe)
HDFS-7579. Improve log reporting during block report rpc failure.
(Charles Lamb via cnauroth)
HDFS-7182. JMX metrics aren't accessible when NN is busy. (Ming Ma via jing9)
HDFS-7596. NameNode should prune dead storages from storageMap.
(Arpit Agarwal via cnauroth)
HDFS-8046. Allow better control of getContentSummary (kihwal)
HDFS-8384. Allow NN to startup if there are files having a lease but are not
under construction. (jing9)
OPTIMIZATIONS
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using
hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang
via Colin P. McCabe)
BUG FIXES
HDFS-7425. NameNode block deletion logging uses incorrect appender.
@ -2223,8 +2119,7 @@ Release 2.6.1 - UNRELEASED
HDFS-7733. NFS: readdir/readdirplus return null directory
attribute on failure. (Arpit Agarwal)
HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P.
McCabe)
HDFS-8486. DN startup may cause severe data loss. (daryn via cmccabe)
HDFS-7213. processIncrementalBlockReport performance degradation.
(Eric Payne via kihwal)
@ -2238,9 +2133,111 @@ Release 2.6.1 - UNRELEASED
HDFS-7225. Remove stale block invalidation work when DN re-registers with
different UUID. (Zhe Zhang and Andrew Wang)
HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade
shutdown command (Eric Payne via kihwal)
HDFS-7575. Upgrade should generate a unique storage ID for each
volume. (Arpit Agarwal)
HDFS-7707. Edit log corruption due to delayed block removal again.
(Yongjun Zhang via kihwal)
HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause
DataNode to register successfully with only one NameNode.(vinayakumarb)
HDFS-7788. Post-2.6 namenode may not start up with an image containing
inodes created with an old release. (Rushabh Shah via kihwal)
HDFS-7009. Active NN and standby NN have different live nodes.
(Ming Ma via cnauroth)
HDFS-7763. fix zkfc hung issue due to not catching exception in a corner
case. (Liang Xie via wang)
HDFS-7871. NameNodeEditLogRoller can keep printing "Swallowing exception"
message. (jing9)
HDFS-7885. Datanode should not trust the generation stamp provided by
client. (Tsz Wo Nicholas Sze via jing9)
HDFS-7610. Fix removal of dynamically added DN volumes (Lei (Eddy) Xu via
Colin P. McCabe)
HDFS-7830. DataNode does not release the volume lock when adding a volume
fails. (Lei Xu via Colin P. Mccabe)
HDFS-7587. Edit log corruption can happen if append fails with a quota
violation. (jing9)
HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade
starts (Zhe Zhang via Colin P. McCabe)
HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu)
HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp
provided by the client is larger than the one stored in the datanode.
(Brahma Reddy Battula via szetszwo)
HDFS-7960. The full block report should prune zombie storages even if
they're not empty. (cmccabe and Eddy Xu via wang)
HDFS-7742. Favoring decommissioning node for replication can cause a block
to stay underreplicated for long periods (Nathan Roberts via kihwal)
HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl
lock for a very long time (sinago via cmccabe)
HDFS-8072. Reserved RBW space is not released if client terminates while
writing block. (Arpit Agarwal)
HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to
finalize upgrade. (jing9)
HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and
fail to tell the DFSClient about it because of a network error (cmccabe)
HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on
post-HDFS-7915 DataNode (cmccabe)
HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart.
(surendra singh lilhore via Xiaoyu Yao)
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query
command is issued. (Brahma Reddy Battula via kihwal)
HDFS-8254. Standby namenode doesn't process DELETED_BLOCK if the add block
request is in edit log. (Rushabh S Shah via kihwal)
HDFS-8404. Pending block replication can get stuck using older genstamp
(Nathan Roberts via kihwal)
HDFS-8431. hdfs crypto class not found in Windows.
(Anu Engineer via cnauroth)
HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits.
(Ming Ma via jing9)
HDFS-8270. create() always retried with hardcoded timeout when file already
exists with open lease (J.Andreina via vinayakumarb)
HDFS-7980. Incremental BlockReport will dramatically slow down namenode
startup. (Walter Su via szetszwo)
HDFS-7446. HDFS inotify should have the ability to determine what txid it
has read up to (cmccabe)
HDFS-8846. Add a unit test for INotify functionality across a layout
version upgrade (Zhe Zhang via Colin P. McCabe)
HDFS-7470. SecondaryNameNode need twice memory when calling
reloadFromImageFile. (zhaoyunjiong via cnauroth)
HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is
flawed. (Kihwal Lee via yliu)
HDFS-7552. Change FsVolumeList toString() to fix
TestDataNodeVolumeFailureToleration (Liang Xie via Colin P. McCabe)
Release 2.6.0 - 2014-11-18
INCOMPATIBLE CHANGES

56
hadoop-mapreduce-project/CHANGES.txt
View File

@ -176,9 +176,6 @@ Release 2.8.0 - UNRELEASED
MAPREDUCE-6349. Fix typo in property org.apache.hadoop.mapreduce.
lib.chain.Chain.REDUCER_INPUT_VALUE_CLASS. (Ray Chiang via ozawa)
MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge
(Gera Shegalov via jlowe)
MAPREDUCE-6165. [JDK8] TestCombineFileInputFormat failed on JDK8.
(Akira AJISAKA via ozawa)
@ -336,14 +333,6 @@ Release 2.7.1 - 2015-07-06
BUG FIXES
MAPREDUCE-6300. Task list sort by task id broken. (Siqi Li via aajisaka)
MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options
which is a regression from MR1 (zxu via rkanter)
MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM
tokens when they roll-over. (Jason Lowe via vinodkv)
MAPREDUCE-6252. JobHistoryServer should not fail when encountering a
missing directory. (Craig Welch via devaraj)
@ -360,10 +349,6 @@ Release 2.7.1 - 2015-07-06
that they don't fail on history-server backed by DFSes with not so strong
guarantees. (Craig Welch via vinodkv)
MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between
copySucceeded() in one thread and copyFailed() in another thread on the
same host. (Junping Du via ozawa)
MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at
the end. (Arun Suresh via kasha)
@ -445,9 +430,6 @@ Release 2.7.0 - 2015-04-20
MAPREDUCE-6248. Exposed the internal MapReduce job's information as a public
API in DistCp. (Jing Zhao via vinodkv)
MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own
class. (Chris Trezzo via kasha)
MAPREDUCE-6263. Configurable timeout between YARNRunner terminate the
application and forcefully kill. (Eric Payne via junping_du)
@ -497,9 +479,6 @@ Release 2.7.0 - 2015-04-20
MAPREDUCE-4879. TeraOutputFormat may overwrite an existing output
directory. (gera)
MAPREDUCE-6166. Reducers do not validate checksum of map outputs when
fetching directly to disk. (Eric Payne via gera)
MAPREDUCE-6045. need close the DataInputStream after open it in
TestMapReduce.java (zxu via rkanter)
@ -509,9 +488,6 @@ Release 2.7.0 - 2015-04-20
MAPREDUCE-3283. mapred classpath CLI does not display the complete classpath
(Varun Saxena via cnauroth)
MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken
service name properly. (Jason Lowe via jianhe)
MAPREDUCE-6231. Grep example job is not working on a fully-distributed
cluster. (aajisaka)
@ -559,9 +535,6 @@ Release 2.7.0 - 2015-04-20
MAPREDUCE-6285. ClientServiceDelegate should not retry upon
AuthenticationException. (Jonathan Eagles via ozawa)
MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal
to a reducer. (Jason Lowe via junping_du)
Release 2.6.2 - UNRELEASED
INCOMPATIBLE CHANGES
@ -578,7 +551,7 @@ Release 2.6.2 - UNRELEASED
cache files so that child processes running hadoop scripts can access these
files. (Junping Du via vinodkv)
Release 2.6.1 - UNRELEASED
Release 2.6.1 - 2015-09-09
INCOMPATIBLE CHANGES
@ -593,6 +566,33 @@ Release 2.6.1 - UNRELEASED
MAPREDUCE-6237. Multiple mappers with DBInputFormat don't work because of
reusing conections. (Kannan Rajah via ozawa)
MAPREDUCE-6166. Reducers do not validate checksum of map outputs when
fetching directly to disk. (Eric Payne via gera)
MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken
service name properly. (Jason Lowe via jianhe)
MAPREDUCE-6303. Read timeout when retrying a fetch error can be fatal
to a reducer. (Jason Lowe via junping_du)
MAPREDUCE-6300. Task list sort by task id broken. (Siqi Li via aajisaka)
MAPREDUCE-6267. Refactor JobSubmitter#copyAndConfigureFiles into it's own
class. (Chris Trezzo via kasha)
MAPREDUCE-6238. MR2 can't run local jobs with -libjars command options
which is a regression from MR1 (zxu via rkanter)
MAPREDUCE-6324. Fixed MapReduce uber jobs to not fail the udpate of AM-RM
tokens when they roll-over. (Jason Lowe via vinodkv)
MAPREDUCE-5649. Reduce cannot use more than 2G memory for the final merge
(Gera Shegalov via jlowe)
MAPREDUCE-6361. NPE issue in shuffle caused by concurrent issue between
copySucceeded() in one thread and copyFailed() in another thread on the
same host. (Junping Du via ozawa)
Release 2.6.0 - 2014-11-18
INCOMPATIBLE CHANGES

390
hadoop-yarn-project/CHANGES.txt
View File

@ -286,9 +286,6 @@ Release 2.8.0 - UNRELEASED
YARN-3581. Deprecate -directlyAccessNodeLabelStore in RMAdminCLI.
(Naganarasimha G R via wangda)
YARN-3700. Made generic history service load a number of latest applications
according to the parameter or the configuration. (Xuan Gong via zjshen)
YARN-3722. Merge multiple TestWebAppUtils into o.a.h.yarn.webapp.util.TestWebAppUtils.
(Masatake Iwasaki via devaraj)
@ -446,9 +443,6 @@ Release 2.8.0 - UNRELEASED
YARN-3459. Fix failiure of TestLog4jWarningErrorMetricsAppender.
(Varun Vasudev via wangda)
YARN-2890. MiniYarnCluster should turn on timeline service if
configured to do so. (Mit Desai via hitesh)
YARN-3266. RMContext#inactiveNodes should have NodeId as map key.
(Chengbing Liu via jianhe)
@ -745,9 +739,6 @@ Release 2.8.0 - UNRELEASED
YARN-4005. Completed container whose app is finished is possibly not
removed from NMStateStore. (Jun Gong via jianhe)
YARN-4047. ClientRMService getApplications has high scheduler lock contention.
(Jason Lowe via jianhe)
YARN-3987. Send AM container completed msg to NM once AM finishes.
(sandflee via jianhe)
@ -792,9 +783,6 @@ Release 2.7.2 - UNRELEASED
YARN-3967. Fetch the application report from the AHS if the RM does not know about it.
(Mit Desai via xgong)
YARN-3978. Configurably turn off the saving of container info in Generic AHS
(Eric Payne via jeagles)
YARN-4092. Fixed UI redirection to print useful messages when both RMs are
in standby mode. (Xuan Gong via jianhe)
@ -828,11 +816,6 @@ Release 2.7.2 - UNRELEASED
YARN-3925. ContainerLogsUtils#getContainerLogFile fails to read container
log files from full disks. (zhihai xu via jlowe)
YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when
Node is connected/disconnected (Bibin A Chundatt via jlowe)
YARN-3999. RM hangs on draing events. (Jian He via xgong)
YARN-3857: Memory leak in ResourceManager with SIMPLE mode.
(mujunchao via zxu)
@ -881,9 +864,6 @@ Release 2.7.1 - 2015-07-06
BUG FIXES
YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when
calling getQueue (Jason Lowe via wangda)
YARN-3462. Patches applied for YARN-2424 are inconsistent between
trunk and branch-2. (Naganarasimha G R via harsh)
@ -910,9 +890,6 @@ Release 2.7.1 - 2015-07-06
YARN-3516. killing ContainerLocalizer action doesn't take effect when
private localizer receives FETCH_FAILURE status.(zhihai xu via xgong)
YARN-3464. Race condition in LocalizerRunner kills localizer before
localizing all resources. (Zhihai Xu via kasha)
YARN-3485. FairScheduler headroom calculation doesn't consider
maxResources for Fifo and FairShare policies. (kasha)
@ -937,10 +914,6 @@ Release 2.7.1 - 2015-07-06
YARN-3434. Interaction between reservations and userlimit can result in
significant ULF violation (tgraves)
YARN-3493. RM fails to come up with error "Failed to load/recover state"
when mem settings are changed. (Jian He via wangda)
<<<<<<< HEAD
YARN-3626. On Windows localized resources are not moved to the front
of the classpath when they should be. (Craig Welch via xgong)
@ -950,20 +923,11 @@ Release 2.7.1 - 2015-07-06
YARN-3537. NPE when NodeManager.serviceInit fails and stopRecoveryStore
invoked (Brahma Reddy Battula via jlowe)
YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when
exceptions happen in stopping NM's sub-services. (Junping Du via jlowe)
YARN-3526. ApplicationMaster tracking URL is incorrectly redirected
on a QJM cluster. (Weiwei Yang via xgong)
YARN-3601. Fix UT TestRMFailover.testRMWebAppRedirect. (Weiwei Yang via xgong)
YARN-3677. Fix findbugs warnings in yarn-server-resourcemanager.
(Vinod Kumar Vavilapalli via ozawa)
YARN-2918. RM should not fail on startup if queue's configured labels do
not exist in cluster-node-labels. (Wangda Tan via jianhe)
YARN-3681. yarn cmd says "could not find main class 'queue'" in windows.
(Craig Welch and Varun Saxena via xgong)
@ -988,18 +952,6 @@ Release 2.7.1 - 2015-07-06
YARN-3686. CapacityScheduler should trim default_node_label_expression.
(Sunil G via wangda)
YARN-2900. Application (Attempt and Container) Not Found in AHS results
in Internal Server Error (500). (Zhijie Shen and Mit Desai via xgong)
YARN-3725. App submission via REST API is broken in secure mode due to
Timeline DT service address is empty. (Zhijie Shen via wangda)
YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM
recovery is enabled (Rohith Sharmaks via jlowe)
YARN-3733. Fix DominantRC#compare() does not work as expected if
cluster resource is empty. (Rohith Sharmaks via wangda)
YARN-3764. CapacityScheduler should forbid moving LeafQueue from one parent
to another. (Wangda Tan via jianhe)
@ -1012,13 +964,6 @@ Release 2.7.1 - 2015-07-06
YARN-3809. Failed to launch new attempts because
ApplicationMasterLauncher's threads all hang (Jun Gong via jlowe)
YARN-3832. Resource Localization fails on a cluster due to existing cache
directories (Brahma Reddy Battula via jlowe)
YARN-3850. NM fails to read files from full disks which can lead to
container logs being lost and other issues (Varun Saxena via jlowe)
Release 2.7.0 - 2015-04-20
INCOMPATIBLE CHANGES
@ -1068,9 +1013,6 @@ Release 2.7.0 - 2015-04-20
YARN-2994. Document work-preserving RM restart. (Jian He via ozawa)
YARN-3249. Add a 'kill application' button to Resource Manager's Web UI.
(Ryu Kobayashi via ozawa)
YARN-2786. Created a yarn cluster CLI and seeded with one command for listing
node-labels collection. (Wangda Tan via vinodkv)
@ -1141,8 +1083,6 @@ Release 2.7.0 - 2015-04-20
YARN-1156. Enhance NodeManager AllocatedGB and AvailableGB metrics
for aggregation of decimal values. (Tsuyoshi OZAWA via junping_du)
YARN-2301. Improved yarn container command. (Naganarasimha G R via jianhe)
YARN-2056. Disable preemption at Queue level (Eric Payne via jlowe)
YARN-2762. Fixed RMAdminCLI to trim and check node-label related arguments
@ -1196,12 +1136,6 @@ Release 2.7.0 - 2015-04-20
YARN-2800. Remove MemoryNodeLabelsStore and add a way to enable/disable
node labels feature. (Wangda Tan via ozawa)
YARN-3024. LocalizerRunner should give DIE action when all resources are
localized. (Chengbing Liu via xgong)
YARN-3092. Created a common ResourceUsage class to track labeled resource
usages in Capacity Scheduler. (Wangda Tan via jianhe)
YARN-3086. Make NodeManager memory configurable in MiniYARNCluster.
(Robert Metzger via ozawa)
@ -1227,9 +1161,6 @@ Release 2.7.0 - 2015-04-20
YARN-3022. Expose Container resource information from NodeManager for
monitoring (adhoot via ranter)
YARN-3098. Created common QueueCapacities class in Capacity Scheduler to
track capacities-by-labels of queues. (Wangda Tan via jianhe)
YARN-3075. NodeLabelsManager implementation to retrieve label to node
mapping (Varun Saxena via wangda)
@ -1270,9 +1201,6 @@ Release 2.7.0 - 2015-04-20
YARN-2079. Recover NonAggregatingLogHandler state upon nodemanager
restart. (Jason Lowe via junping_du)
YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track
capacities-by-label. (Wangda Tan via jianhe)
YARN-3158. Correct log messages in ResourceTrackerService.
(Varun Saxena via xgong)
@ -1297,8 +1225,6 @@ Release 2.7.0 - 2015-04-20
YARN-2799. Cleanup TestLogAggregationService based on the change in YARN-90.
(Zhihai Xu via junping_du)
YARN-3230. Clarify application states on the web UI. (Jian He via wangda)
YARN-3237. AppLogAggregatorImpl fails to log error cause.
(Rushabh S Shah via xgong)
@ -1335,9 +1261,6 @@ Release 2.7.0 - 2015-04-20
YARN-3122. Metrics for container's actual CPU usage.
(Anubhav Dhoot via kasha)
YARN-1809. Synchronize RM and TimeLineServer Web-UIs. (Zhijie Shen and
Xuan Gong via jianhe)
YARN-2190. Added CPU and memory limit options to the default container
executor for Windows containers. (Chuan Liu via jianhe)
@ -1372,56 +1295,26 @@ Release 2.7.0 - 2015-04-20
YARN-2713. "RM Home" link in NM should point to one of the RMs in an
HA setup. (kasha)
YARN-2766. Made ApplicationHistoryManager return a sorted list of apps,
attempts and containers. (Robert Kanter via zjshen)
YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state
on app recovery. (Rohith Sharmaks via jianhe)
YARN-2857. ConcurrentModificationException in ContainerLogAppender
(Mohammad Kamrul Islam via jlowe)
YARN-2816. NM fail to start with NPE during container recovery (Zhihai Xu
via jlowe)
YARN-2432. RMStateStore should process the pending events before close.
(Varun Saxena via jianhe)
YARN-1703. Fixed ResourceManager web-proxy to close connections correctly.
(Rohith Sharma via vinodkv)
YARN-2414. RM web UI: app page will crash if app is failed before any
attempt has been created (Wangda Tan via jlowe)
YARN-2870. Updated the command to run the timeline server in the document.
(Masatake Iwasaki via zjshen)
YARN-2878. Fix DockerContainerExecutor.apt.vm formatting. (Abin Shahab via
jianhe)
YARN-2865. Fixed RM to always create a new RMContext when transtions from
StandBy to Active. (Rohith Sharmaks via jianhe)
YARN-2315. FairScheduler: Set current capacity in addition to capacity.
(Zhihai Xu via kasha)
YARN-1984. LeveldbTimelineStore does not handle db exceptions properly
(Varun Saxena via jlowe)
YARN-2697. Remove useless RMAuthenticationHandler. (Haosong Huang via zjshen)
YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users.
(Jason Lowe via jianhe)
YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated
log file is corrupted (Varun Saxena via jlowe)
YARN-2894. Fixed a bug regarding application view acl when RM fails over.
(Rohith Sharmaks via jianhe)
YARN-2874. Dead lock in "DelegationTokenRenewer" which blocks RM to execute
any further apps. (Naganarasimha G R via kasha)
YARN-2461. Fix PROCFS_USE_SMAPS_BASED_RSS_ENABLED property in
YarnConfiguration. (rchiang via rkanter)
@ -1434,18 +1327,12 @@ Release 2.7.0 - 2015-04-20
YARN-2931. PublicLocalizer may fail until directory is initialized by
LocalizeRunner. (Anubhav Dhoot via kasha)
YARN-2910. FSLeafQueue can throw ConcurrentModificationException.
(Wilfred Spiegelenburg via kasha)
YARN-2930. Fixed TestRMRestart#testRMRestartRecoveringNodeLabelManager
intermittent failure. (Wangda Tan via jianhe)
YARN-2924. Fixed RMAdminCLI to not convert node labels to lower case.
(Wangda Tan via jianhe)
YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher
(Rohith Sharmaks via jianhe)
YARN-2243. Order of arguments for Preconditions.checkNotNull() is wrong in
SchedulerApplicationAttempt ctor. (devaraj)
@ -1463,15 +1350,9 @@ Release 2.7.0 - 2015-04-20
YARN-2944. InMemorySCMStore can not be instantiated with ReflectionUtils#newInstance.
(Chris Trezzo via kasha)
YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie)
(Jian He via jlowe)
YARN-2675. containersKilled metrics is not updated when the container is killed
during localization. (Zhihai Xu via kasha)
YARN-2952. Fixed incorrect version check in StateStore. (Rohith Sharmaks
via jianhe)
YARN-2975. FSLeafQueue app lists are accessed without required locks. (kasha)
YARN-2977. Fixed intermittent TestNMClient failure.
@ -1479,12 +1360,6 @@ Release 2.7.0 - 2015-04-20
YARN-2939. Fix new findbugs warnings in hadoop-yarn-common. (Li Lu via junping_du)
YARN-2920. Changed CapacityScheduler to kill containers on nodes where
node labels are changed. (Wangda Tan via jianhe)
YARN-2340. Fixed NPE when queue is stopped during RM restart.
(Rohith Sharmaks via jianhe)
YARN-2940. Fix new findbugs warnings in rest of the hadoop-yarn components. (Li Lu
via junping_du)
@ -1496,9 +1371,6 @@ Release 2.7.0 - 2015-04-20
YARN-2988. Graph#save() may leak file descriptors. (Ted Yu via ozawa)
YARN-2992. ZKRMStateStore crashes due to session expiry. (Karthik Kambatla
via jianhe)
YARN-2938. Fixed new findbugs warnings in hadoop-yarn-resourcemanager and
hadoop-yarn-applicationhistoryservice. (Varun Saxena via zjshen)
@ -1508,15 +1380,9 @@ Release 2.7.0 - 2015-04-20
YARN-2991. Fixed DrainDispatcher to reuse the draining code path in
AsyncDispatcher. (Rohith Sharmaks via zjshen)
YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue.
(Rohith Sharmaks via ozawa)
YARN-2958. Made RMStateStore not update the last sequence number when updating the
delegation token. (Varun Saxena via zjshen)
YARN-2978. Fixed potential NPE while getting queue info. (Varun Saxena via
jianhe)
YARN-2230. Fixed few configs description in yarn-default.xml. (Vijay Bhat
via jianhe)
@ -1526,18 +1392,12 @@ Release 2.7.0 - 2015-04-20
YARN-2936. Changed YARNDelegationTokenIdentifier to set proto fields on
getProto method. (Varun Saxena via jianhe)
YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed
container statuses on heartbeat. (Chengbing Liu via jianhe)
YARN-3014. Replaces labels on a host should update all NM's labels on that
host. (Wangda Tan via jianhe)
YARN-3027. Scheduler should use totalAvailable resource from node instead of
availableResource for maxAllocation. (adhoot via rkanter)
YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler
when activating applications. (Craig Welch via jianhe)
YARN-2861. Fixed Timeline DT secret manager to not reuse RM's configs.
(Zhijie Shen via jianhe)
@ -1568,21 +1428,12 @@ Release 2.7.0 - 2015-04-20
YARN-3088. LinuxContainerExecutor.deleteAsUser can throw NPE if native
executor returns an error (Eric Payne via jlowe)
YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService
might lead NM to crash. (Varun Saxena via jianhe)
YARN-3103. AMRMClientImpl does not update AMRM token properly. (Jason Lowe
via jianhe)
YARN-3079. Scheduler should also update maximumAllocation when updateNodeResource.
(Zhihai Xu via wangda)
YARN-3029. FSDownload.unpack() uses local locale for FS case conversion, may not
work everywhere. (Varun Saxena via ozawa)
YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage
to track used-resources-by-label.(Wangda Tan via jianhe)
YARN-3077. Fixed RM to create zk root path recursively. (Chun Chen via jianhe)
YARN-3113. Release audit warning for Sorting icons.psd. (stevel via kihwal)
@ -1608,18 +1459,12 @@ Release 2.7.0 - 2015-04-20
YARN-1537. Fix race condition in
TestLocalResourcesTrackerImpl.testLocalResourceCache. (xgong via acmurthy)
YARN-2694. Ensure only single node label specified in ResourceRequest.
(Wangda Tan via jianhe)
YARN-3089. LinuxContainerExecutor does not handle file arguments to
deleteAsUser (Eric Payne via jlowe)
YARN-3143. RM Apps REST API can return NPE or entries missing id and other
fields (jlowe)
YARN-3094. Reset timer for liveness monitors after RM recovery. (Jun Gong
via jianhe)
YARN-2971. RM uses conf instead of token service address to renew timeline
delegation tokens (jeagles)
@ -1629,10 +1474,6 @@ Release 2.7.0 - 2015-04-20
YARN-2809. Implement workaround for linux kernel panic when removing
cgroup (Nathan Roberts via jlowe)
YARN-2246. Made the proxy tracking URL always be
http(s)://proxy addr:port/proxy/<appId> to avoid duplicate sections. (Devaraj
K via zjshen)
YARN-3160. Fix non-atomic operation on nodeUpdateQueue in RMNodeImpl.
(Chengbing Liu via junping_du)
@ -1663,9 +1504,6 @@ Release 2.7.0 - 2015-04-20
YARN-2749. Fix some testcases from TestLogAggregationService fails in trunk.
(Xuan Gong via junping_du)
YARN-3207. Secondary filter matches entites which do not have the key being
filtered for. (Zhijie Shen via xgong)
YARN-3132. RMNodeLabelsManager should remove node from node-to-label mapping
when node becomes deactivated. (Wangda Tan via jianhe)
@ -1678,15 +1516,9 @@ Release 2.7.0 - 2015-04-20
YARN-3194. RM should handle NMContainerStatuses sent by NM while
registering if NM is Reconnected node (Rohith via jlowe)
YARN-3238. Connection timeouts to nodemanagers are retried at
multiple levels (Jason Lowe via xgong)
YARN-3247. TestQueueMappings should use CapacityScheduler explicitly.
(Zhihai Xu via ozawa)
YARN-3239. WebAppProxy does not support a final tracking url which has
query fragments and params (Jian He via jlowe)
YARN-3256. TestClientToAMTokens#testClientTokenRace is not running against
all Schedulers even when using ParameterizedSchedulerTestBase.
(Anubhav Dhoot via devaraj)
@ -1697,27 +1529,12 @@ Release 2.7.0 - 2015-04-20
YARN-3265. Fixed a deadlock in CapacityScheduler by always passing a queue's
available resource-limit from the parent queue. (Wangda Tan via vinodkv)
YARN-3222. Fixed RMNode to send scheduler events in sequential order when a
node reconnects. (Rohith Sharma K S via jianhe)
YARN-3131. YarnClientImpl should check FAILED and KILLED state in
submitApplication (Chang Li via jlowe)
YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending
jobs. (Siqi Li via kasha)
YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving
events for old client. (Zhihai Xu via kasha)
YARN-3227. Timeline renew delegation token fails when RM user's TGT is expired
(Zhijie Shen via xgong)
YARN-3275. CapacityScheduler: Preemption happening on non-preemptable
queues (Eric Payne via jlowe)
YARN-3287. Made TimelineClient put methods do as the correct login context.
(Daryn Sharp and Jonathan Eagles via zjshen)
YARN-3300. Outstanding_resource_requests table should not be shown in AHS.
(Xuan Gong via jianhe)
@ -1732,9 +1549,6 @@ Release 2.7.0 - 2015-04-20
YARN-3154. Added additional APIs in LogAggregationContext to avoid aggregating
running logs of application when rolling is enabled. (Xuan Gong via vinodkv)
YARN-3267. Timelineserver applies the ACL rules after applying the limit on
the number of records (Chang Li via jeagles)
YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't work
in ATS / RM web ui. (Naganarasimha G R via xgong)
@ -1747,18 +1561,12 @@ Release 2.7.0 - 2015-04-20
YARN-3379. Fixed missing data in localityTable and ResourceRequests table
in RM WebUI. (Xuan Gong via jianhe)
YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die.
(Brahma Reddy Battula via wangda)
YARN-3384. TestLogAggregationService.verifyContainerLogs fails after
YARN-2777. (Naganarasimha G R via ozawa)
YARN-3336. FileSystem memory leak in DelegationTokenRenewer.
(Zhihai Xu via cnauroth)
YARN-3393. Getting application(s) goes wrong when app finishes before
starting the attempt. (Zhijie Shen via xgong)
YARN-2213. Change proxy-user cookie log in AmIpFilter to DEBUG.
(Varun Saxena via xgong)
@ -1772,9 +1580,6 @@ Release 2.7.0 - 2015-04-20
YARN-3466. Fix RM nodes web page to sort by node HTTP-address, #containers
and node-label column (Jason Lowe via wangda)
YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token
renewal of applications part of a bigger workflow. (Daryn Sharp via vinodkv)
Release 2.6.2 - UNRELEASED
INCOMPATIBLE CHANGES
@ -1787,18 +1592,211 @@ Release 2.6.2 - UNRELEASED
BUG FIXES
Release 2.6.1 - UNRELEASED
Release 2.6.1 - 2015-09-09
INCOMPATIBLE CHANGES
NEW FEATURES
YARN-3249. Add a 'kill application' button to Resource Manager's Web UI.
(Ryu Kobayashi via ozawa)
IMPROVEMENTS
YARN-3230. Clarify application states on the web UI. (Jian He via wangda)
YARN-1809. Synchronize RM and TimeLineServer Web-UIs. (Zhijie Shen and
Xuan Gong via jianhe)
YARN-3092. Created a common ResourceUsage class to track labeled resource
usages in Capacity Scheduler. (Wangda Tan via jianhe)
YARN-3098. Created common QueueCapacities class in Capacity Scheduler to
track capacities-by-labels of queues. (Wangda Tan via jianhe)
YARN-2301. Improved yarn container command. (Naganarasimha G R via jianhe)
YARN-3978. Configurably turn off the saving of container info in Generic AHS
(Eric Payne via jeagles)
OPTIMIZATIONS
BUG FIXES
YARN-2856. Fixed RMAppImpl to handle ATTEMPT_KILLED event at ACCEPTED state
on app recovery. (Rohith Sharmaks via jianhe)
YARN-2816. NM fail to start with NPE during container recovery (Zhihai Xu
via jlowe)
YARN-2414. RM web UI: app page will crash if app is failed before any
attempt has been created (Wangda Tan via jlowe)
YARN-2865. Fixed RM to always create a new RMContext when transtions from
StandBy to Active. (Rohith Sharmaks via jianhe)
YARN-2906. CapacitySchedulerPage shows HTML tags for a queue's Active Users.
(Jason Lowe via jianhe)
YARN-2905. AggregatedLogsBlock page can infinitely loop if the aggregated
log file is corrupted (Varun Saxena via jlowe)
YARN-2890. MiniYARNCluster should start the timeline server based on the
configuration. (Mit Desai via zjshen)
YARN-2894. Fixed a bug regarding application view acl when RM fails over.
(Rohith Sharmaks via jianhe)
YARN-2874. Dead lock in "DelegationTokenRenewer" which blocks RM to execute
any further apps. (Naganarasimha G R via kasha)
YARN-2910. FSLeafQueue can throw ConcurrentModificationException.
(Wilfred Spiegelenburg via kasha)
YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher
(Rohith Sharmaks via jianhe)
YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie)
(Jian He via jlowe)
YARN-1984. LeveldbTimelineStore does not handle db exceptions properly
(Varun Saxena via jlowe)
YARN-2952. Fixed incorrect version check in StateStore. (Rohith Sharmaks
via jianhe)
YARN-2340. Fixed NPE when queue is stopped during RM restart.
(Rohith Sharmaks via jianhe)
YARN-2992. ZKRMStateStore crashes due to session expiry. (Karthik Kambatla
via jianhe)
YARN-2922. ConcurrentModificationException in CapacityScheduler's LeafQueue.
(Rohith Sharmaks via ozawa)
YARN-2997. Fixed NodeStatusUpdater to not send alreay-sent completed
container statuses on heartbeat. (Chengbing Liu via jianhe)
YARN-3011. Possible IllegalArgumentException in ResourceLocalizationService
might lead NM to crash. (Varun Saxena via jianhe)
YARN-3103. AMRMClientImpl does not update AMRM token properly. (Jason Lowe
via jianhe)
YARN-3094. Reset timer for liveness monitors after RM recovery. (Jun Gong
via jianhe)
YARN-2246. Made the proxy tracking URL always be
http(s)://proxy addr:port/proxy/<appId> to avoid duplicate sections. (Devaraj
K via zjshen)
YARN-3207. Secondary filter matches entites which do not have the key being
filtered for. (Zhijie Shen via xgong)
YARN-3238. Connection timeouts to nodemanagers are retried at
multiple levels (Jason Lowe via xgong)
YARN-3239. WebAppProxy does not support a final tracking url which has
query fragments and params (Jian He via jlowe)
YARN-3222. Fixed RMNode to send scheduler events in sequential order when a
node reconnects. (Rohith Sharma K S via jianhe)
YARN-3231. FairScheduler: Changing queueMaxRunningApps interferes with pending
jobs. (Siqi Li via kasha)
YARN-3242. Asynchrony in ZK-close can lead to ZKRMStateStore watcher receiving
events for old client. (Zhihai Xu via kasha)
YARN-3227. Timeline renew delegation token fails when RM user's TGT is expired
(Zhijie Shen via xgong)
YARN-3287. Made TimelineClient put methods do as the correct login context.
(Daryn Sharp and Jonathan Eagles via zjshen)
YARN-3267. Timelineserver applies the ACL rules after applying the limit on
the number of records (Chang Li via jeagles)
YARN-3369. Missing NullPointer check in AppSchedulingInfo causes RM to die.
(Brahma Reddy Battula via wangda)
YARN-3393. Getting application(s) goes wrong when app finishes before
starting the attempt. (Zhijie Shen via xgong)
YARN-3055. Fixed ResourceManager's DelegationTokenRenewer to not stop token
renewal of applications part of a bigger workflow. (Daryn Sharp via vinodkv)
YARN-3493. RM fails to come up with error "Failed to load/recover state"
when mem settings are changed. (Jian He via wangda)
YARN-3487. CapacityScheduler scheduler lock obtained unnecessarily when
calling getQueue (Jason Lowe via wangda)
YARN-3024. LocalizerRunner should give DIE action when all resources are
localized. (Chengbing Liu via xgong)
YARN-3464. Race condition in LocalizerRunner kills localizer before
localizing all resources. (Zhihai Xu via kasha)
YARN-3641. NodeManager: stopRecoveryStore() shouldn't be skipped when
exceptions happen in stopping NM's sub-services. (Junping Du via jlowe)
YARN-3526. ApplicationMaster tracking URL is incorrectly redirected
on a QJM cluster. (Weiwei Yang via xgong)
YARN-2766. Made ApplicationHistoryManager return a sorted list of apps,
attempts and containers. (Robert Kanter via zjshen)
YARN-3700. Made generic history service load a number of latest applications
according to the parameter or the configuration. (Xuan Gong via zjshen)
YARN-2900. Application (Attempt and Container) Not Found in AHS results
in InternalServer Error (500). (Zhijie Shen and Mit Desai via xgong)
YARN-3725. App submission via REST API is broken in secure mode due to
Timeline DT service address is empty. (Zhijie Shen via wangda)
YARN-3585. NodeManager cannot exit on SHUTDOWN event triggered and NM
recovery is enabled (Rohith Sharmaks via jlowe)
YARN-3832. Resource Localization fails on a cluster due to existing cache
directories (Brahma Reddy Battula via jlowe)
YARN-3850. NM fails to read files from full disks which can lead to
container logs being lost and other issues (Varun Saxena via jlowe)
YARN-3990. AsyncDispatcher may overloaded with RMAppNodeUpdateEvent when
Node is connected/disconnected (Bibin A Chundatt via jlowe)
YARN-2637. Fixed max-am-resource-percent calculation in CapacityScheduler
when activating applications. (Craig Welch via jianhe)
YARN-3733. Fix DominantRC#compare() does not work as expected if
cluster resource is empty. (Rohith Sharmaks via wangda)
YARN-2920. Changed CapacityScheduler to kill containers on nodes where
node labels are changed. (Wangda Tan via jianhe)
YARN-2978. Fixed potential NPE while getting queue info. (Varun Saxena via
jianhe)
YARN-3099. Capacity Scheduler LeafQueue/ParentQueue should use ResourceUsage
to track used-resources-by-label.(Wangda Tan via jianhe)
YARN-2694. Ensure only single node label specified in ResourceRequest.
(Wangda Tan via jianhe)
YARN-3124. Fixed CS LeafQueue/ParentQueue to use QueueCapacities to track
capacities-by-label. (Wangda Tan via jianhe)
YARN-2918. RM should not fail on startup if queue's configured labels do
not exist in cluster-node-labels. (Wangda Tan via jianhe)
YARN-3999. RM hangs on draing events. (Jian He via xgong)
YARN-4047. ClientRMService getApplications has high scheduler lock contention.
(Jason Lowe via jianhe)
Release 2.6.0 - 2014-11-18
INCOMPATIBLE CHANGES