Vinod Kumar Vavilapalli
b99a85139e
Creating 2.6.2 entries in CHANGES.txt files.
...
(cherry picked from commit 33474939a0
)
2015-09-25 16:00:06 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
41d19f4740
Release process for 2.6.1: Set the release date for 2.6.1
2015-09-23 15:15:36 -07:00
Vinod Kumar Vavilapalli
b4d876d837
Set the release date for 2.6.1-RC1
2015-09-16 12:04:57 -07:00
Vinod Kumar Vavilapalli
075d899d10
Set the release date for 2.6.1-RC0
2015-09-09 15:39:02 -07:00
Colin Patrick Mccabe
adc3405321
HDFS-7552. Change FsVolumeList toString() to fix TestDataNodeVolumeFailureToleration (Liang Xie via Colin P. McCabe)
...
(cherry picked from commit a4876c130f
)
(cherry picked from commit 22f72c7946
)
2015-09-09 14:59:31 -07:00
yliu
0adcfe6c7e
HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu)
...
(cherry picked from commit 146db49f7f
)
2015-09-08 22:57:36 -07:00
cnauroth
c3928b3068
HDFS-7570. SecondaryNameNode need twice memory when calling reloadFromImageFile. Contributed by zhaoyunjiong.
...
(cherry picked from commit 85aec75ce5
)
(cherry picked from commit 27eecaef10
)
2015-09-08 17:21:58 -07:00
Vinod Kumar Vavilapalli
33537078a8
HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao.
2015-09-08 16:45:38 -07:00
Colin Patrick Mccabe
a976acc02d
HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe)
...
(cherry picked from commit a4d9acc51d
)
(cherry picked from commit 9264b7e119
)
2015-09-08 16:18:30 -07:00
Colin Patrick Mccabe
2e86d1f254
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe)
...
(cherry picked from commit 7b424f938c
)
(cherry picked from commit cbd11681ce
)
2015-09-08 16:12:45 -07:00
Colin Patrick Mccabe
4363145128
HDFS-7446. HDFS inotify should have the ability to determine what txid it has read up to (cmccabe)
...
(cherry picked from commit 75a326aaff
)
(cherry picked from commit 06552a15d5
)
2015-09-08 15:32:17 -07:00
Tsz-Wo Nicholas Sze
8ed162bcbf
HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su
...
(cherry picked from commit 4e1f2eb395
)
2015-09-07 11:46:33 -07:00
Vinayakumar B
a2434dcf2a
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
...
(cherry picked from commit 54f83d9bd9
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
(cherry picked from commit 8090a6ee63
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit 04a7000c8d955a134d86967f4b4622ea920e1ea7)
2015-09-03 14:06:09 -07:00
Jing Zhao
46b9393cab
HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma.
...
(cherry picked from commit 7817674a3a
)
(cherry picked from commit 17fb442a4c
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
(cherry picked from commit fad2a062ddbb955a42dd5a90d64781617287f8df)
2015-09-03 13:40:45 -07:00
cnauroth
563dbd29eb
HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer.
...
(cherry picked from commit 50eeea1300
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
(cherry picked from commit 25db341278
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
(cherry picked from commit 77a10e76e99c14cd26ebb3664304f6ed9cc7bf65)
2015-09-03 12:59:35 -07:00
Kihwal Lee
fb096b97cc
HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts.
...
(cherry picked from commit 8860e352c3
)
(cherry picked from commit 536b9ee6d6
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestPendingReplication.java
(cherry picked from commit 2d5e60fa12a62463cd54f1b6b0fcb2ccdbd82c42)
2015-09-03 12:32:49 -07:00
Kihwal Lee
b3ca4dfaff
HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
...
(cherry picked from commit 2d4ae3d18b
)
(cherry picked from commit f264a5aeed
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockReplacement.java
(cherry picked from commit 470019e9b88e0fcede926442b91d102b595c7ace)
2015-09-03 12:19:21 -07:00
Colin Patrick Mccabe
51d5fc62c2
HDFS-7278. Add a command that allows sysadmins to manually trigger full block reports from a DN (cmccabe)
...
(cherry picked from commit baf794dc40
)
(cherry picked from commit 5f3d967aae
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
(cherry picked from commit a776ef5ad2876b9acf6cf89824c306783f7759f1)
2015-09-03 11:43:27 -07:00
Kihwal Lee
bbed4e67a9
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 6f622672b6
)
(cherry picked from commit 802a5775f3
)
(cherry picked from commit 995382c5234ad6c07f327e5d1f2a1c7e391a0b60)
2015-09-03 11:38:46 -07:00
Xiaoyu Yao
08006b84f8
HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao)
...
(cherry picked from commit 0100b15501
)
(cherry picked from commit e68e8b3b5c
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
(cherry picked from commit b054cb68fa0fc6d1e9e77ac84575731e7d1ec0c7)
2015-09-02 15:31:46 -07:00
Colin Patrick Mccabe
181281cfca
HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe)
...
(cherry picked from commit a8898445dc
)
(cherry picked from commit 3d0385c3c8
)
2015-09-02 14:37:51 -07:00
Colin Patrick Mccabe
9df4b7c10e
HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe)
...
(cherry picked from commit bc9cb3e271
)
(cherry picked from commit c71b54fa32
)
2015-09-02 14:20:17 -07:00
Jing Zhao
b7e7896410
HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao.
...
(cherry picked from commit fddd55279d
)
(cherry picked from commit 38b031d6ba
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ha/BootstrapStandby.java
(cherry picked from commit b4e227e65d3a81150f46aac5e987d759defc1452)
2015-09-02 11:06:04 -07:00
Kihwal Lee
619f793846
HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee.
...
(cherry picked from commit 285b31e75e
)
(cherry picked from commit 7e622076d4
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/ContentSummaryComputationContext.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
(cherry picked from commit 1ef5e0b18066ca949adcf4c55a41f186c47e7264)
2015-09-01 21:24:53 -07:00
Arpit Agarwal
31d30e8111
HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal)
...
(cherry picked from commit f0324738c9
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/extdataset/ExternalReplicaInPipeline.java
(cherry picked from commit de21de7e2243ef8a89082121d838b88e3c10f05b)
2015-09-01 20:40:09 -07:00
Colin Patrick Mccabe
c3f5ea11ec
HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe)
...
(cherry picked from commit 28bebc81db
)
(cherry picked from commit a827089905
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit c3a3092c37926eca75ea149c4c061742f6599b40)
2015-09-01 20:15:20 -07:00
Kihwal Lee
bc8728cd27
HDFS-7742. Favoring decommissioning node for replication can cause a block to stay
...
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48
)
(cherry picked from commit c4cedfc1d6
)
(cherry picked from commit c6b68a82adea8de488b255594d35db8e01f5fc8f)
2015-09-01 19:04:52 -07:00
Andrew Wang
0b1e66f01d
HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu.
...
(cherry picked from commit 50ee8f4e67
)
(cherry picked from commit 2f46ee50bd
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocol/TestBlockListAsLongs.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestTriggerBlockReport.java
(cherry picked from commit 03d4af39e794dc03d764122077b434d658b6405e)
2015-09-01 18:30:18 -07:00
Tsz-Wo Nicholas Sze
9b105b78d4
HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula
...
(cherry picked from commit fe693b72de
)
(cherry picked from commit 4c648779d69a668f5147b183af2d40d45d1227d2)
2015-09-01 18:25:02 -07:00
yliu
553efd7190
HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu)
...
(cherry picked from commit 90164ffd84
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFileTruncate.java
(cherry picked from commit 2c9a7461ec2ceba5885e95bc79f8dcbfd198df60)
2015-09-01 17:06:37 -07:00
Colin Patrick Mccabe
81a445edf8
HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe)
...
(cherry picked from commit 43b41f2241
)
(cherry picked from commit 219eb22c15
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NNUpgradeUtil.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
(cherry picked from commit 03798416bfe27383c52e4d9f632fe9fa168c6e95)
2015-09-01 16:55:50 -07:00
Jing Zhao
49ef26b5ac
HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao.
...
Committed Ming Ma's 2.6 patch.
(cherry picked from commit 7f0bb5d3fe0db2e6b9354c8d8a1b603f2390184f)
2015-09-01 16:33:30 -07:00
Colin Patrick Mccabe
9642a861e1
HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe)
...
(cherry picked from commit 5c1036d598
)
(cherry picked from commit eefca23e8c
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetTestUtil.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
(cherry picked from commit c723f3b1bd9eab261ab5edca33c4dae5ce3d0d30)
2015-09-01 15:35:08 -07:00
Colin Patrick Mccabe
5dbb0325df
HDFS-7610. Fix removal of dynamically added DN volumes (Lei (Eddy) Xu via Colin P. McCabe)
...
(cherry picked from commit a17584936c
)
(cherry picked from commit 7779f38e68
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
(cherry picked from commit 65ae3e2ff16ce1114a0115ff916837b0173b77f1)
2015-09-01 15:29:59 -07:00
Jing Zhao
b25491dc45
HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze.
...
(cherry picked from commit 24db0812be
)
(cherry picked from commit 994dadb9ba
)
(cherry picked from commit 0bc5c6495a7feb4365af0ce5fe48fc87b7e1749f)
2015-09-01 15:17:57 -07:00
Jing Zhao
6e090bc53d
HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contributed by Jing Zhao.
...
(cherry picked from commit b442aeec95
)
(cherry picked from commit 6090f51725
)
(cherry picked from commit e1af1ac4e91d36b21df18ce5627e1f69f27f0776)
2015-09-01 13:36:21 -07:00
Andrew Wang
406ec495bb
HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie.
...
(cherry picked from commit 7105ebaa9f
)
(cherry picked from commit efb7e287f4
)
(cherry picked from commit fd70e4db105e140fc3d60042abb3f598c9afd13f)
2015-09-01 11:30:04 -07:00
cnauroth
be4d4c9cbf
HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma.
...
(cherry picked from commit 769507bd7a
)
(cherry picked from commit 657a6e389b
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
(cherry picked from commit d5ddc3450f2f49ea411de590ff3de15b5ec4e17c)
2015-09-01 11:23:26 -07:00
Kihwal Lee
02e0b6e306
HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah.
...
(cherry picked from commit 7ae5255a16
)
(cherry picked from commit b9157f92fc
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/resources/image-with-zero-block-size.tar.gz
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImage.java
(cherry picked from commit 1faa44d8f4d7b944e99dd0470ea2638c7653a131)
2015-08-31 18:32:00 -07:00
Vinayakumar B
6154abfeda
HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode.(Contributed by Vinayakumar B)
...
(cherry picked from commit 3d15728ff5
)
(cherry picked from commit a1bf7aecf7
)
(cherry picked from commit c1e65de57e8ef760586e28cd37397ea9a7ac7944)
2015-08-31 17:26:28 -07:00
Kihwal Lee
a1c3e6a058
HDFS-7707. Edit log corruption due to delayed block removal again. Contributed by Yongjun Zhang
...
(cherry picked from commit 843806d03a
)
(cherry picked from commit c428d303f6
)
(cherry picked from commit 21d8b22feef2fc9f6793d9c157391b142368e588)
2015-08-31 15:55:07 -07:00
Arpit Agarwal
f81f97eb7c
HDFS-7575. Upgrade should generate a unique storage ID for each volume. (Contributed by Arpit Agarwal)
...
(cherry picked from commit 1d9d166c0b
)
(cherry picked from commit ca8e1b0739b6653833f9bc8990ab126420703f66)
2015-08-31 15:08:42 -07:00
Kihwal Lee
ae288f7291
HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade shutdown command. Contributed by Eric Payne.
...
(cherry picked from commit 6bbf9fdd04
)
(cherry picked from commit 33534a0c9a
)
(cherry picked from commit e9a28251ee46e64e1b99b2dd54b0432bdc0b9578)
2015-08-31 11:53:24 -07:00
cnauroth
8382e52c94
HDFS-7596. NameNode should prune dead storages from storageMap. Contributed by Arpit Agarwal.
...
(cherry picked from commit ef3c3a832c
)
(cherry picked from commit 75e4e55e12
)
(cherry picked from commit cc637d6ece64dfeb89e78c7e9766836149e098be)
2015-08-31 11:26:48 -07:00
Jing Zhao
c8ce11d067
HDFS-7182. JMX metrics aren't accessible when NN is busy. Contributed by Ming Ma.
...
(cherry picked from commit 4b589e7cfa
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSNamesystemMBean.java
(cherry picked from commit 96f0813c5d6140aabe7b2837f30971936276e689)
2015-08-30 21:21:05 -07:00
cnauroth
e6b52fc3e9
HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb.
...
(cherry picked from commit 7e2d9a3242
)
(cherry picked from commit f0acb7c2a2
)
(cherry picked from commit 33fb7b45195a89b6464e4a1cb3fbf6bbad2bcecb)
2015-08-30 21:17:34 -07:00
Colin Patrick Mccabe
c2e00e0012
HDFS-7531. Improve the concurrent access on FsVolumeList (Lei Xu via Colin P. McCabe)
...
(cherry picked from commit 3b173d9517
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit dda1fc169d
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeList.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsDatasetImpl.java
(cherry picked from commit ba28192f9d5a8385283bd717bca494e6981d378f)
2015-08-30 20:11:41 -07:00
Colin Patrick Mccabe
70f33f5eff
HDFS-7035. Make adding a new data directory to the DataNode an atomic operation and improve error handling (Lei Xu via Colin P. McCabe)
...
(cherry picked from commit a9331fe9b0
)
(cherry picked from commit ec2621e907
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
(cherry picked from commit d79a5849999cdb0bc315938b80ed71b4f2dcb720)
2015-08-30 12:13:33 -07:00
Kihwal Lee
8549891cb3
HDFS-7213. processIncrementalBlockReport performance degradation.
...
Contributed by Eric Payne.
(cherry picked from commit e226b5b40d
)
(cherry picked from commit 946463efef
)
(cherry picked from commit f94aa4d25c2f96faf5164e807c2c3eb031e9a1fe)
2015-08-30 11:46:16 -07:00
Andrew Wang
0898c21e24
HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang)
...
(cherry picked from commit 406c09ad11
)
(cherry picked from commit 2e15754a92
)
(cherry picked from commit 014d07de2e9b39be4b6793f0e09fcf8548570ad5)
2015-08-27 18:48:17 -07:00