Akira Ajisaka
ce01a5951b
HDFS-9317. Document fsck -blockId and -storagepolicy options in branch-2.7. (aajisaka)
2015-10-28 13:14:58 +09:00
Kihwal Lee
59a2072135
HDFS-8950. NameNode refresh doesn't remove DataNodes that are no longer in the allowed list. Contributed by Daniel Templeton.
2015-10-27 16:42:33 -05:00
Kihwal Lee
1d23e1ec07
HDFS-7725. Incorrect 'nodes in service' metrics caused all writes to fail. Contributed by Ming Ma.
...
(cherry picked from commit 8104d52269
)
2015-10-27 11:09:05 -05:00
Arpit Agarwal
653ef52ef2
HDFS-9305. Delayed heartbeat processing causes storm of subsequent heartbeats. (Contributed by Arpit Agarwal)
2015-10-26 15:47:37 -07:00
Kihwal Lee
f2fd011cc1
HDFS-9290. DFSClient#callAppend() is not backward compatible for slightly older NameNodes. Contributed by Tony Wu.
...
(cherry picked from commit b9e0417bdf
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
2015-10-23 16:46:14 -05:00
Kihwal Lee
4cf7f8441a
HDFS-9220. Reading small file (< 512 bytes) that is open for append fails due to incorrect checksum. Contributed by Jing Zhao.
...
(cherry picked from commit c7c36cbd62
)
2015-10-15 11:26:14 -05:00
Kihwal Lee
efba749760
HDFS-8676. Delayed rolling upgrade finalization can cause heartbeat expiration. Contributed by Walter Su.
...
(cherry picked from commit 5b43db47a3
)
2015-10-13 13:25:10 -05:00
Kihwal Lee
61388b41e7
HDFS-8656. Preserve compatibility of ClientProtocol#rollingUpgrade after finalization.
...
(cherry picked from commit 60b858bfa6
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRollingUpgrade.java
2015-10-13 10:46:20 -05:00
Kihwal Lee
3a5b9f49df
HDFS-7645. Rolling upgrade is restoring blocks from trash multiple times (Contributed by Vinayakumar B and Keisuke Ogiwara)
...
(cherry picked from commit abf3ad988d
)
2015-10-13 10:29:42 -05:00
Kihwal Lee
5cb80992fb
HDFS-9178. Slow datanode I/O can cause a wrong node to be marked bad. Contributed by Kihwal Lee.
...
(cherry picked from commit 99e5204ff5
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientProtocolForPipelineRecovery.java
2015-10-07 10:24:55 -05:00
Kihwal Lee
b2bc72a051
HDFS-8850. VolumeScanner thread exits with exception if there is no block pool to be scanned but there are suspicious blocks. (Colin Patrick McCabe via yliu)
...
(cherry picked from commit d2b941f94a
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-10-02 14:59:29 -05:00
Kihwal Lee
6e7c76a5bd
HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee.
2015-09-28 15:19:57 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
2b4c7fcc67
Release process for 2.6.1: Set the release date for 2.6.1
...
(cherry picked from commit 1f707ecffd
)
(cherry picked from commit f48af332a7
)
2015-09-23 15:13:09 -07:00
Vinayakumar B
d800e2e9ed
HDFS-9043. Doc updation for commands in HDFS Federation (Contributed by J.Andreina)
...
(cherry picked from commit 57003fa971
)
(cherry picked from commit 96e3fbf401
)
2015-09-22 16:40:19 +05:30
Vinayakumar B
7d3e882a49
HDFS-9042. Update document for the Storage policy name (Contributed by J.Andreina)
...
(cherry picked from commit d8455479b8
)
2015-09-12 12:37:01 +05:30
Kihwal Lee
f2cc6b5fdf
HDFS-8099. Change "DFSInputStream has been closed already" message to debug log level (Charles Lamb via Colin P. McCabe)
...
(cherry picked from commit 30acb7372a
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-09-11 15:59:29 -05:00
Vinod Kumar Vavilapalli
fb15053124
HDFS-7314. Moving to 2.6.1 CHANGES.txt section.
...
(cherry picked from commit f103a70af5
)
2015-09-10 16:48:53 -07:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
c05ba079e3
Updating all CHANGES.txt files to move entires from future releases into 2.6.1
...
section given the large number of backports to 2.6.1.
(cherry picked from commit fbbb7ff1ed
)
(cherry picked from commit cbf90f93f9
)
2015-09-10 16:10:56 -07:00
Colin Patrick Mccabe
ee830b0d27
HDFS-8846. Add a unit test for INotify functionality across a layout version upgrade (Zhe Zhang via Colin P. McCabe)
...
(cherry picked from commit a4d9acc51d
)
(cherry picked from commit 9264b7e119
)
2015-09-10 11:56:30 -07:00
Ming Ma
decf8a6653
HDFS-7314. When the DFSClient lease cannot be renewed, abort open-for-write files rather than the entire DFSClient. (mingma)
...
(cherry picked from commit fbd88f1062
)
(cherry picked from commit 516bbf1c20
)
2015-09-10 11:56:29 -07:00
Jing Zhao
6c127b44ca
HDFS-7609. Avoid retry cache collision when Standby NameNode loading edits. Contributed by Ming Ma.
...
(cherry picked from commit 7817674a3a
)
(cherry picked from commit 17fb442a4c
)
2015-09-10 11:56:28 -07:00
cnauroth
188096f1fe
HDFS-8431. hdfs crypto class not found in Windows. Contributed by Anu Engineer.
...
(cherry picked from commit 50eeea1300
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/bin/hdfs.cmd
(cherry picked from commit 25db341278
)
2015-09-10 11:56:27 -07:00
Xiaoyu Yao
d3562b4e1d
HDFS-8219. setStoragePolicy with folder behavior is different after cluster restart. (surendra singh lilhore via Xiaoyu Yao)
...
(cherry picked from commit 0100b15501
)
(cherry picked from commit e68e8b3b5c
)
2015-09-10 11:56:22 -07:00
Kihwal Lee
cfb27d7f78
HDFS-8046. Allow better control of getContentSummary. Contributed by Kihwal Lee.
...
(cherry picked from commit 285b31e75e
)
(cherry picked from commit 7e622076d4
)
2015-09-10 10:53:19 -07:00
Akira Ajisaka
c5a74c5c37
HDFS-9033. dfsadmin -metasave prints "NaN" for cache used%. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 0113e4528d
)
(cherry picked from commit eb9e06f8e3
)
2015-09-09 17:23:25 +09:00
Jing Zhao
fd100ccbae
HDFS-8384. Allow NN to startup if there are files having a lease but are not under construction. Contributed by Jing Zhao.
2015-09-08 10:52:42 -07:00
Tsuyoshi Ozawa
139fd21c83
HADOOP-5323. Trash documentation should describe its directory structure and configurations. Contributed by Weiwei Yang.
...
(cherry picked from commit 9a87f81ce6
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsDesign.md
2015-09-03 22:39:12 +09:00
yliu
3ab43accaf
HDFS-8995. Flaw in registration bookeeping can make DN die on reconnect. (Kihwal Lee via yliu)
2015-09-02 09:00:22 +08:00
Kihwal Lee
285dad32eb
fix merge error in CHANGES.TXT
2015-08-28 15:37:07 -05:00
Kihwal Lee
42aa880df3
HDFS-8879. Quota by storage type usage incorrectly initialized upon namenode restart. Contributed by Xiaoyu Yao.
...
(cherry picked from commit 3e715a4f4c
)
2015-08-28 13:27:17 -05:00
Jing Zhao
ba0f66984c
HDFS-8891. HDFS concat should keep srcs order. Contributed by Yong Zhang.
...
(cherry picked from commit dc7a061668
)
(cherry picked from commit 2edfc3ce71
)
2015-08-20 20:42:20 -07:00
Vinod Kumar Vavilapalli
9faee040f9
Creating 2.6.2 entries in CHANGES.txt files.
...
(cherry picked from commit 33474939a0
)
2015-08-20 17:39:29 -07:00
yliu
d4b6c4a3cd
HDFS-8863. The remaining space check in BlockPlacementPolicyDefault is flawed. (Kihwal Lee via yliu)
2015-08-20 20:43:26 +08:00
Jing Zhao
4c005a4e16
HDFS-8867. Enable optimized block reports. Contributed by Daryn Sharp.
...
(cherry picked from commit f61120d964
)
(cherry picked from commit c0a4cd978a
)
2015-08-19 10:41:30 -07:00
Akira Ajisaka
f98b5ae3bb
HDFS-8852. HDFS architecture documentation of version 2.x is outdated about append write support. Contributed by Ajith S.
...
(cherry picked from commit fc509f66d8
)
(cherry picked from commit 38b344adfa
)
2015-08-18 23:33:28 +09:00
Vinayakumar B
4d94bc1d95
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
...
Moved to 2.6.1
(cherry picked from commit 84bf71295a
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-08-14 16:15:10 +05:30
Vinayakumar B
90f3641728
HDFS-7225. Remove stale block invalidation work when DN re-registers with different UUID. (Zhe Zhang and Andrew Wang)
...
Moved to 2.6.1
(cherry picked from commit 08bd4edf40
)
2015-08-14 12:38:59 +05:30
Vinayakumar B
b06d342749
HDFS-7263. Snapshot read can reveal future bytes for appended files. Contributed by Tao Luo.
...
Moved CHANGES.txt entry to 2.6.1
(cherry picked from commit fa2641143c
)
2015-08-14 11:46:58 +05:30
Vinayakumar B
f40714f8d8
HDFS-7235. DataNode#transferBlock should report blocks that don't exist using reportBadBlock (yzhang via cmccabe)
...
Moved CHANGES.txt entry to 2.6.1
(cherry picked from commit f2b4bc9b6a
)
2015-08-14 11:38:28 +05:30
Vinayakumar B
5ba4065345
HDFS-7213. processIncrementalBlockReport performance degradation. Contributed by Eric Payne.
...
Moved CHANGES.TXT entry to 2.6.1
(cherry picked from commit d25cb8fe12
)
(cherry picked from commit 2c0063d420
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-08-14 11:27:46 +05:30
Akira Ajisaka
b919c1c044
HDFS-8806. Inconsistent metrics: number of missing blocks with replication factor 1 not properly cleared. Contributed by Zhe Zhang.
...
(cherry picked from commit 206d4933a5
)
(cherry picked from commit b2a532f61c
)
2015-07-24 18:38:44 +09:00
Akira Ajisaka
3e793224f9
HDFS-6945. BlockManager should remove a block from excessReplicateMap and decrement ExcessBlocks metric when the block is removed. (aajisaka)
...
(cherry picked from commit 18a91fe4df
)
(cherry picked from commit b85bbca745
)
2015-07-21 11:42:21 +09:00
Kihwal Lee
e263219829
HDFS-8722. Optimize datanode writes for small writes and flushes. Contributed by Kihwal Lee
...
(cherry picked from commit 59388a8015
)
2015-07-14 14:08:15 -05:00
Vinod Kumar Vavilapalli (I am also known as @tshooter.)
dcafd3ccd2
Release process for 2.7.1: Set the release date for 2.7.1.
...
(cherry picked from commit bf89ddb9b8
)
2015-07-06 16:41:38 -07:00
Yongjun Zhang
849b47525d
HDFS-8659. Block scanner INFO message is spamming logs. Contributed by Yongjun Zhang.
...
(cherry picked from commit 43a1288fff
)
2015-06-29 14:56:14 -07:00
Vinod Kumar Vavilapalli
a36f090c55
Preparing for 2.7.2 development: mvn versions:set -DnewVersion=2.7.2
2015-06-28 16:41:12 -07:00
Vinod Kumar Vavilapalli
2e99675c72
Adding release 2.7.2 to CHANGES.txt.
...
(cherry picked from commit aad6a7d5db
)
2015-06-28 16:33:49 -07:00
Arpit Agarwal
f6550361f7
HDFS-8681. BlockScanner is incorrectly disabled by default. (Contributed by Arpit Agarwal)
2015-06-28 14:58:07 -07:00
Arpit Agarwal
4a8db40f3b
HDFS-8626. Add test case for branch-2.7
2015-06-23 10:03:44 -07:00
Colin Patrick Mccabe
86c01d90ba
HDFS-8480. Fix performance and timeout issues in HDFS-7929 by using hard-links to preserve old edit logs, instead of copying them. (Zhe Zhang via Colin P. McCabe)
...
(cherry picked from commit 7b424f938c
)
(cherry picked from commit cbd11681ce
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSUpgrade.java
2015-06-22 14:46:57 -07:00
Arpit Agarwal
25b99e4816
HDFS-7164. Feature documentation for HDFS-6581. (Contributed by Arpit Agarwal)
2015-06-20 13:08:18 -07:00
Allen Wittenauer
0adb6d8292
HDFS-7546. Document, and set an accepting default for dfs.namenode.kerberos.principal.pattern (Harsh J via aw)
...
(cherry picked from commit 63613c79c1
)
2015-06-20 07:45:39 -07:00
Arpit Agarwal
dc4171cb66
HDFS-8626. Reserved RBW space is not released if creation of RBW File fails. (Contributed by kanaka kumar avvaru)
2015-06-19 18:30:53 -07:00
Yongjun Zhang
8c03a079df
HDFS-8633. Fix setting of dfs.datanode.readahead.bytes in hdfs-default.xml to match DFSConfigKeys. Contributed by Ray Chiang.
...
(cherry picked from commit b42f1ec13d
)
2015-06-19 09:51:48 -07:00
Kihwal Lee
6de4d52c31
HDFS-4660. Block corruption can happen during pipeline recovery. Contributed by Kihwal Lee.
...
(cherry picked from commit c74517c46b
)
2015-06-16 15:44:38 -05:00
Xiaoyu Yao
fe35ba1df9
HDFS-8597. Fix TestFSImage#testZeroBlockSize on Windows. Contributed by Xiaoyu Yao.
...
(cherry picked from commit 4e88ff5b27
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-06-16 12:51:34 -07:00
Yongjun Zhang
f4ae56d605
HDFS-8544. Incorrect port specified in HFTP Guide document in branch-2. (Contributed by Brahma Reddy Battula)
...
(cherry picked from commit b1d3ed02c1
)
2015-06-15 21:51:21 -07:00
Tsz-Wo Nicholas Sze
632cb542f6
HDFS-8361. Choose SSD over DISK in block placement.
2015-06-15 17:29:33 -07:00
Tsz-Wo Nicholas Sze
7a248f2e9e
HDFS-8576. Lease recovery should return true if the lease can be released and the file can be closed. Contributed by J.Andreina
2015-06-15 16:13:56 -07:00
Arpit Agarwal
a1810d75f0
HDFS-8600. TestWebHdfsFileSystemContract.testGetFileBlockLocations fails in branch-2.7. (Contributed by Arpit Agarwal)
2015-06-14 20:18:31 -07:00
Arpit Agarwal
1145d09e0a
HDFS-8595. TestCommitBlockSynchronization fails in branch-2.7. (Patch applies to all branches). (Contributed by Arpit Agarwal)
2015-06-14 16:24:42 -07:00
Yongjun Zhang
6dda57a817
HDFS-8596. TestDistributedFileSystem et al tests are broken in branch-2 due to incorrect setting of "datanode" attribute. Contributed by Yongjun Zhang.
...
(cherry picked from commit b0dc291961
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/DatanodeHttpServer.java
(cherry picked from commit cd578fa289
)
2015-06-14 12:04:16 -07:00
Haohui Mai
3651feb49d
HDFS-8572. DN always uses HTTP/localhost@REALM principals in SPNEGO. Contributed by Haohui Mai.
2015-06-11 19:14:48 -07:00
Arpit Agarwal
3ec9b63586
HDFS-8583. Document that NFS gateway does not work with rpcbind on SLES 11. (Arpit Agarwal)
2015-06-11 15:08:15 -07:00
Colin Patrick Mccabe
0bcc68337b
HDFS-8566. HDFS documentation about debug commands wrongly identifies them as "hdfs dfs" commands (Surendra Singh Lilhore via Colin P. McCabe)
...
(cherry picked from commit 587309bbba
)
(cherry picked from commit 754dd3496c
)
2015-06-11 14:30:21 -07:00
Arpit Agarwal
105cadb421
HADOOP-12078. The default retry policy does not handle RetriableException correctly. (Contributed by Arpit Agarwal)
2015-06-09 20:58:55 -07:00
Xiaoyu Yao
8c1a46ca4f
HDFS-8522. Change heavily recorded NN logs from INFO to DEBUG level. Contributed by xyao.
2015-06-05 17:13:40 -07:00
Vinayakumar B
dd98cfd328
HDFS-8523. Remove usage information on unsupported operation 'fsck -showprogress' from branch-2 (Contributed by J.Andreina)
...
(cherry picked from commit 0ed9c2d8fe
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSCommands.md
2015-06-03 15:18:42 +05:30
Vinayakumar B
8090a6ee63
HDFS-8270. create() always retried with hardcoded timeout when file already exists with open lease (Contributed by J.Andreina)
...
(cherry picked from commit 54f83d9bd9
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/NameNodeProxies.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileCreation.java
2015-06-03 12:20:54 +05:30
Colin Patrick Mccabe
a7058caa72
HDFS-8521. Add VisibleForTesting annotation to BlockPoolSlice#selectReplicaToDelete. (cmccabe)
...
(cherry picked from commit 806e407ac8
)
(cherry picked from commit 12497b0543
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/BlockPoolSlice.java
2015-06-02 20:07:35 -07:00
Colin Patrick Mccabe
323ffccfe9
HDFS-8486. DN startup may cause severe data loss (Daryn Sharp via Colin P. McCabe)
...
(cherry picked from commit 03fb5c6425
)
(cherry picked from commit 17e369511d
)
2015-06-02 11:46:12 -07:00
Kihwal Lee
1ff3f16ed0
HDFS-5215. dfs.datanode.du.reserved is not considered while computing
...
available space ( Brahma Reddy Battula via Yongjun Zhang)
(cherry picked from commit 66763bb06f
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-06-02 08:14:57 -05:00
Xiaoyu Yao
c550825673
HDFS-8451. DFSClient probe for encryption testing interprets empty URI property for enabled. Contributed by Steve Loughran.
...
(cherry picked from commit 05e04f34f2
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSUtil.java
2015-05-21 12:49:49 -07:00
Kihwal Lee
7ac66f8712
HDFS-8404. Pending block replication can get stuck using older genstamp. Contributed by Nathan Roberts.
...
(cherry picked from commit 8860e352c3
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
2015-05-19 13:17:45 -05:00
Tsz-Wo Nicholas Sze
59d1b4a323
HDFS-8405. Fix a typo in NamenodeFsck. Contributed by Takanobu Asanuma
2015-05-19 03:17:51 +08:00
cnauroth
82530ba0f1
HADOOP-11966. Variable cygwin is undefined in hadoop-config.sh when executed through hadoop-daemon.sh. Contributed by Chris Nauroth.
...
(cherry picked from commit ddc4afeff2
)
2015-05-13 12:26:02 -07:00
Tsz-Wo Nicholas Sze
2113e0a3f2
HDFS-8143. Mover should exit after some retry when failed to move blocks. Contributed by surendra singh lilhore
2015-05-13 11:59:01 -07:00
Vinayakumar B
ef2488e7f0
HDFS-6300. Prevent multiple balancers from running simultaneously (Contributed by Rakesh R)
...
(cherry picked from commit 065d8f2a34
)
2015-05-13 17:30:03 +05:30
Kihwal Lee
1e4a686ea9
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop. Contributed by Rushabh Shah.
...
(cherry picked from commit ea11590aad
)
2015-05-11 17:11:51 -05:00
Kihwal Lee
f57b1bfbdd
HDFS-8245. Standby namenode doesn't process DELETED_BLOCK if the addblock request is in edit log. Contributed by Rushabh S Shah.
...
(cherry picked from commit 2d4ae3d18b
)
2015-05-08 16:38:43 -05:00
Kihwal Lee
bbb9f6aafe
HDFS-7894. Rolling upgrade readiness is not updated in jmx until query command is issued. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 6f622672b6
)
2015-05-08 09:32:44 -05:00
Vinayakumar B
1fbefe586b
HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina)
...
Added one missed change, missed while resolving conflict
(cherry picked from commit 3ee1b50dad
)
2015-05-08 15:42:17 +05:30
Vinayakumar B
99309a9fec
HDFS-8226. Non-HA rollback compatibility broken (Contributed by J.Andreina)
...
(cherry picked from commit e01c1a979c
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/bin/start-dfs.sh
hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HdfsUserGuide.md
(cherry picked from commit 3a894b4563
)
2015-05-08 15:42:02 +05:30
Tsz-Wo Nicholas Sze
74bd9666d1
HDFS-7980. Incremental BlockReport will dramatically slow down namenode startup. Contributed by Walter Su
2015-05-07 11:38:15 -07:00
Colin Patrick Mccabe
e056e0a3c1
HDFS-8305: HDFS INotify: the destination field of RenameOp should always end with the file name (cmccabe)
...
(cherry picked from commit fcd4cb7516
)
(cherry picked from commit 1f01d8347a
)
2015-05-05 11:07:13 -07:00
Vinayakumar B
69d837428c
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B)
...
Reverted earlier commit
(cherry picked from commit 318081ccd7
)
2015-05-05 11:07:59 +05:30
Colin Patrick Mccabe
472fd563e4
HDFS-8213. DFSClient should use hdfs.client.htrace HTrace configuration prefix rather than hadoop.htrace (cmccabe)
...
(cherry picked from commit b82567d455
)
(cherry picked from commit 9edea9507d
)
2015-05-01 11:52:08 -07:00
Arun Suresh
4e6ff8c78b
HDFS-8091: ACLStatus and XAttributes should be presented to INodeAttributesProvider before returning to client (asuresh)
...
(cherry picked from commit 922b7ed21d
)
2015-05-01 11:42:29 -07:00
Akira Ajisaka
185a1ff275
HDFS-7770. Need document for storage type label of data node storage locations under dfs.data.dir. Contributed by Xiaoyu Yao.
...
(cherry picked from commit de9404f02f
)
(cherry picked from commit fdb886d656
)
2015-04-30 20:13:47 +09:00
Haohui Mai
6ac2b5712b
HDFS-8269. getBlockLocations() does not resolve the .reserved path and generates incorrect edit logs when updating the atime. Contributed by Haohui Mai.
2015-04-29 11:13:03 -07:00
Haohui Mai
4b9147ef8c
HDFS-8273. FSNamesystem#Delete() should not call logSync() when holding the lock. Contributed by Haohui Mai.
2015-04-28 18:06:22 -07:00
Tsz-Wo Nicholas Sze
6b20b325f4
HDFS-8204. Mover/Balancer should not schedule two replicas to the same datanode. Contributed by Walter Su
2015-04-28 13:07:20 -07:00
Colin Patrick Mccabe
176d22b440
HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe)
...
(cherry picked from commit a8898445dc
)
(cherry picked from commit 3d0385c3c8
)
2015-04-23 19:28:38 -07:00
Colin Patrick Mccabe
c2e6d2589a
HADOOP-11802. Fix renamed constant in test file in backport to 2.7.1 (cmccabe)
2015-04-23 19:16:28 -07:00
Colin Patrick Mccabe
496afb5e1a
HADOOP-11802. DomainSocketWatcher thread terminates sometimes after there is an I/O error during requestShortCircuitShm (cmccabe)
...
(cherry picked from commit a0e0a63209
)
(cherry picked from commit 788b76761d
)
2015-04-23 19:09:50 -07:00
Tsz-Wo Nicholas Sze
5b4eda7573
HDFS-8147. StorageGroup in Dispatcher should override equals nad hashCode. Contributed by surendra singh lilhore
2015-04-23 11:57:16 -07:00
Arpit Agarwal
b17d365fa1
HDFS-8163. Using monotonicNow for block report scheduling causes test failures on recently restarted systems. (Arpit Agarwal)
2015-04-21 11:17:27 -07:00
Vinod Kumar Vavilapalli
7dd5f42a76
Set the release date for 2.7.0
...
(cherry picked from commit d52de61544
)
2015-04-20 20:18:38 -07:00
Vinayakumar B
1c8d54bc04
HDFS-7916. 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop (Contributed by Vinayakumar B)
...
(cherry picked from commit 867d5d2675
)
2015-04-21 07:45:59 +05:30
Arpit Agarwal
cc42ad03d9
HDFS-8179. DFSClient#getServerDefaults returns null within 1 hour of system start. (Contributed by Xiaoyu Yao)
2015-04-20 15:42:58 -07:00
Jitendra Pandey
0743f128a0
HDFS-8153. Error Message points to wrong parent directory in case of path component name length error. Contributed by Anu Engineer.
...
cherry picked from 369ddc67bd
2015-04-16 22:23:35 -07:00
Akira Ajisaka
cffc801ecb
Revert "Merge r1588258 from trunk, HDFS-6153. Document "fileId" and "childrenNum" fields in the FileStatus Json schema"
...
This reverts commit 641cbc0feb
.
2015-04-16 16:46:19 -05:00
Akira Ajisaka
35e1936f84
HDFS-8149. The footer of the Web UI "Hadoop, 2014" is old. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit de0f1700c1
)
(cherry picked from commit 6d183561ff
)
2015-04-15 15:32:29 -05:00
Jing Zhao
e1ce0700eb
HDFS-7934. Update RollingUpgrade rollback documentation: should use bootstrapstandby for standby NN. Contributed by J. Andreina.
...
(cherry picked from commit b172d03595
)
(cherry picked from commit 725621d95a
)
2015-04-15 13:11:45 -07:00
Jing Zhao
9e11ae684a
HDFS-8151. Always use snapshot path as source when invalid snapshot names are used for diff based distcp. Contributed by Jing Zhao.
...
(cherry picked from commit 4c097e473b
)
(cherry picked from commit d4dd97eabd
)
2015-04-15 12:41:34 -07:00
Jing Zhao
50c8d06314
HDFS-8127. NameNode Failover during HA upgrade can cause DataNode to finalize upgrade. Contributed by Jing Zhao.
...
(cherry picked from commit fddd55279d
)
(cherry picked from commit 38b031d6ba
)
2015-04-14 15:15:36 -07:00
Arun Suresh
0a95b4f8aa
HDFS-7931. DistributedFileSystem should not look for keyProvider in cache if Encryption is disabled (asuresh)
...
(cherry picked from commit 7660da95cb
)
2015-04-10 11:43:49 -07:00
Konstantin V Shvachko
e964bbc856
HDFS-8081. Split getAdditionalBlock() into two methods. Contributed by Konstantin Shvachko
2015-04-09 22:04:48 -07:00
Vinod Kumar Vavilapalli
a7fa46c2e0
Preparing for 2.7.1 development: mvn versions:set
2015-04-09 14:13:30 -07:00
Vinod Kumar Vavilapalli
2b44fcaa53
Adding release 2.7.1 to CHANGES.txt
...
(cherry picked from commit 623fd46c1e
)
2015-04-09 14:01:23 -07:00
Colin Patrick Mccabe
4d5b1fbdec
HDFS-8063: Fix intermittent test failures in TestTracing (Masatake Iwasaki via Colin P. McCabe)
...
(cherry picked from commit 61dc2ea3fe
)
(cherry picked from commit 8dac245920
)
2015-04-09 11:42:29 -07:00
Kihwal Lee
9a111fcd1d
HDFS-8025. Addendum fix for HDFS-3087 Decomissioning on NN restart can complete without blocks being replicated. Contributed by Ming Ma.
...
(cherry picked from commit 5a540c3d31
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-04-09 10:01:44 -05:00
Haohui Mai
84ba4db2a0
Revert HDFS-7813.
2015-04-08 16:03:35 -07:00
Haohui Mai
eb53d6b967
Revert HDFS-7808.
2015-04-08 16:00:26 -07:00
Arpit Agarwal
12739b541b
HDFS-8072. Reserved RBW space is not released if client terminates while writing block. (Arpit Agarwal)
2015-04-08 11:38:42 -07:00
cnauroth
5021f52f7b
HDFS-8038. PBImageDelimitedTextWriter#getEntry output HDFS path in platform-specific format. Contributed by Xiaoyu Yao.
...
(cherry picked from commit 672ed462965ea7da10ce3df48c02e2a3bd13f0ae)
(cherry picked from commit 950dc1b1995191327e6b919ee17a87e59b749264)
2015-04-07 13:33:38 -07:00
Konstantin V Shvachko
2cb9dac9a3
HDFS-8071. Redundant checkFileProgress() in PART II of getAdditionalBlock(). Contributed by Konstantin Shvachko.
2015-04-06 23:35:01 -07:00
Colin Patrick Mccabe
3f1d1d0b48
HDFS-7999. FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time (sinago via cmccabe)
...
(cherry picked from commit 28bebc81db
)
(cherry picked from commit a827089905
)
2015-04-06 09:10:34 -07:00
Vinod Kumar Vavilapalli
5387a289fc
HADOOP-11776. Fixed the broken JDiff support in Hadoop 2. Contributed by Li Lu.
...
(cherry picked from commit 4b3948ea36
)
2015-04-04 13:53:34 -07:00
Vinod Kumar Vavilapalli
996b0ca173
HADOOP-11377. Added Null.java without which jdiff completely flops. Contributed by Tsuyoshi Ozawa.
...
(cherry picked from commit 5370e7128b
)
2015-04-04 13:48:58 -07:00
Colin Patrick Mccabe
aa5eab8ec4
HDFS-8051. Fix missing import statement from backport.
2015-04-03 19:07:25 -07:00
Colin Patrick Mccabe
54b88943ff
HDFS-8051. FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. (Lei (Eddy) Xu via Colin P. McCabe)
...
(cherry picked from commit ef591b1d6a
)
(cherry picked from commit b26ba22a90
)
2015-04-03 16:57:06 -07:00
Colin Patrick Mccabe
520043b438
HDFS-7996. After swapping a volume, BlockReceiver reports ReplicaNotFoundException (Lei (Eddy) Xu via Colin P. McCabe)
...
(cherry picked from commit 023133cef9
)
(cherry picked from commit 0c5069c432
)
2015-04-03 14:27:09 -07:00
cnauroth
6855bbc6ca
HDFS-8039. Fix TestDebugAdmin#testRecoverLease and testVerfiyBlockChecksumCommand on Windows. Contributed by Xiaoyu Yao.
...
(cherry picked from commit db80e42891
)
(cherry picked from commit 28e0602e7b
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-04-03 10:45:15 -07:00
Jing Zhao
b08bc43c9b
HDFS-7811. Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage. Contributed by Xiaoyu Yao and Jing Zhao.
...
(cherry picked from commit 72f6bd4893
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit 2c1f33d178
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-04-02 21:27:32 -07:00
Tsz-Wo Nicholas Sze
acfe44e5b0
HDFS-7954. TestBalancer#testBalancerWithPinnedBlocks should not be executed on Windows. Contributed by Xiaoyu Yao
2015-04-02 11:14:04 -07:00
Brandon Li
a445f8b675
HDFS-8001 RpcProgramNfs3 : wrong parsing of dfs.blocksize. Contributed by Remi Catherinot
...
(cherry picked from commit 4d14816c26
)
(cherry picked from commit 27970b3f46
)
2015-04-01 17:30:00 -07:00
Haohui Mai
8ebbbc6ebd
HDFS-8036. Use snapshot path as source when using snapshot diff report in DistCp. Contributed by Jing Zhao.
2015-04-01 16:51:25 -07:00
Haohui Mai
86c0c6b044
HDFS-7748. Separate ECN flags from the Status in the DataTransferPipelineAck. Contributed by Anu Engineer and Haohui Mai.
2015-03-30 12:16:25 -07:00
Kihwal Lee
a0ed29a058
HDFS-7742. Favoring decommissioning node for replication can cause a block to stay
...
underreplicated for long periods. Contributed by Nathan Roberts.
(cherry picked from commit 04ee18ed48
)
2015-03-30 10:11:47 -05:00
Haohui Mai
d481a8d79b
HADOOP-11748. The secrets of auth cookies should not be specified in configuration in clear text. Contributed by Li Lu and Haohui Mai.
2015-03-26 17:35:54 -07:00
Colin Patrick Mccabe
9411cf9b65
HDFS-7410. Support CreateFlags with append() to support hsync() for appending streams (Vinayakumar B via Colin P. McCabe)
...
(cherry picked from commit 61df1b27a7
)
(cherry picked from commit 2a750c9aa6
)
2015-03-26 13:29:17 -07:00
Arpit Agarwal
c808d5fe55
HDFS-7824. GetContentSummary API and its namenode implementation for Storage Type Quota/Usage. (Contributed by Xiaoyu Yao)
2015-03-26 10:48:03 -07:00
Kihwal Lee
370c91461f
HDFS-7963. Fix expected tracing spans in TestTracing. Contributed by Masatake Iwasaki.
2015-03-26 08:45:39 -05:00
Jitendra Pandey
d286673c60
HDFS-6826. Plugin interface to enable delegation of HDFS authorization assertions. Contributed by Arun Suresh.
...
(cherry picked from commit 456cec127b
)
2015-03-24 16:17:06 -07:00
Brandon Li
14d1cccfad
HDFS-7977. NFS couldn't take percentile intervals. Contributed by Brandon Li
...
(cherry picked from commit 570a83ae80
)
(cherry picked from commit 260dbe96c3
)
2015-03-24 10:53:20 -07:00
Brandon Li
cc5922607a
HDFS-7976. Update NFS user guide for mount option 'sync' to minimize or avoid reordered writes. Contributed by Brandon Li
...
(cherry picked from commit 51f1f49374
)
(cherry picked from commit 95bde88986
)
2015-03-24 10:30:19 -07:00
Andrew Wang
a7f447d460
HDFS-7961. Trigger full block report after hot swapping disk. Contributed by Eddy Xu.
...
(cherry picked from commit 6413d34986
)
2015-03-24 09:07:27 -07:00
Plamen Jeliazkov
da318cbbde
HDFS-7956. Improve logging for DatanodeRegistration. Contributed by Plamen Jeliazkov.
2015-03-23 23:09:52 -07:00
Andrew Wang
af0af28afc
HDFS-7960. The full block report should prune zombie storages even if they're not empty. Contributed by Colin McCabe and Eddy Xu.
...
(cherry picked from commit 50ee8f4e67
)
2015-03-23 22:01:37 -07:00
Tsz-Wo Nicholas Sze
87079cde7d
HDFS-7884. Fix NullPointerException in BlockSender when the generation stamp provided by the client is larger than the one stored in the datanode. Contributed by Brahma Reddy Battula
2015-03-24 13:52:18 +09:00
cnauroth
8e1c33e703
HDFS-7917. Use file to replace data dirs in test to simulate a disk failure. Contributed by Lei (Eddy) Xu.
...
(cherry picked from commit 2c238ae4e0
)
(cherry picked from commit 01c0bcb176
)
2015-03-23 16:30:48 -07:00
Akira Ajisaka
2742f12b58
HDFS-7881. TestHftpFileSystem#testSeek fails in branch-2. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit fad8c78173
)
2015-03-24 06:25:21 +09:00
Brandon Li
6b9f2d9f39
HDFS-7942. NFS: support regexp grouping in nfs.exports.allowed.hosts. Contributed by Brandon Li
...
(cherry picked from commit 36af4a913c
)
(cherry picked from commit 503d8e4164
)
2015-03-23 10:13:11 -07:00
Kihwal Lee
08f2f7ed34
HDFS-6841. Use Time.monotonicNow() wherever applicable instead of Time.now(). Contributed by Vinayakumar B
...
(cherry picked from commit 99a8dcd19528b265d4fda9ae09a17e4af52f2782)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/DatanodeInfo.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
(cherry picked from commit 63b7870aec007081c82427587cb6e1d38e2c70f8)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
2015-03-20 14:06:02 -05:00
Jing Zhao
03fb9b4ce0
HDFS-7957. Truncate should verify quota before making changes. Contributed by Jing Zhao.
...
(cherry picked from commit d368d3647a
)
(cherry picked from commit 6be52e42a9
)
2015-03-20 11:56:39 -07:00
yliu
faeb8b9b53
HDFS-7962. Remove duplicated logs in BlockManager. (yliu)
2015-03-20 05:32:19 +08:00
yliu
0da3fcb4b0
HDFS-7930. commitBlockSynchronization() does not remove locations. (yliu)
2015-03-19 23:26:26 +08:00
Kihwal Lee
4e967d0bcd
HDFS-7932. Speed up the shutdown of datanode during rolling upgrade. Contributed by Kihwal Lee.
...
(cherry picked from commit 61a4c7fc98
)
2015-03-19 12:29:16 -05:00
Vinayakumar B
76ce2e8ea0
HDFS.7849. Update documentation for enabling a new feature in rolling upgrade ( Contributed by J.Andreina )
...
(cherry picked from commit 5be5129d93
)
2015-03-19 19:10:23 +05:30
Kihwal Lee
b8f269af9d
HDFS-7816. Unable to open webhdfs paths with "+". Contributed by Haohui Mai
...
(cherry picked from commit e79be0ee12
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit ceb39c1cc6
)
2015-03-19 08:04:19 -05:00
Colin Patrick Mccabe
9e61835678
HDFS-7929. inotify unable fetch pre-upgrade edit log segments once upgrade starts (Zhe Zhang via Colin P. McCabe)
...
(cherry picked from commit 43b41f2241
)
(cherry picked from commit 219eb22c15
)
2015-03-18 19:08:53 -07:00
Jing Zhao
0bf549728d
HDFS-7587. Edit log corruption can happen if append fails with a quota violation. Contributed by Jing Zhao.
...
(cherry picked from commit c7c71cdba5
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
(cherry picked from commit 5a5b244648
)
2015-03-18 19:00:06 -07:00
Jing Zhao
4b8de154aa
HDFS-7943. Append cannot handle the last block with length greater than the preferred block size. Contributed by Jing Zhao.
...
(cherry picked from commit bee5a6a64a
)
(cherry picked from commit 6dcc79507d
)
2015-03-18 18:47:21 -07:00
Colin Patrick Mccabe
94976cb369
HDFS-7054. Make DFSOutputStream tracing more fine-grained (cmccabe)
...
(cherry picked from commit 8234fd0e10
)
(cherry picked from commit 79c07bbaca
)
2015-03-18 18:15:34 -07:00
Haohui Mai
388696c089
HDFS-7945. The WebHdfs system on DN does not honor the length parameter. Contributed by Haohui Mai.
2015-03-18 16:20:07 -07:00
Arpit Agarwal
bb986988d4
HDFS-7914. TestJournalNode#testFailToStartWithBadConfig fails when the default dfs.journalnode.http-address port 8480 is in use. (Contributed by Xiaoyu Yao)
2015-03-18 14:05:37 -07:00
Haohui Mai
482aeee784
HDFS-7697. Mark the PB OIV tool as experimental. Contributed by Lei (Eddy) Xu.
2015-03-18 13:34:21 -07:00
Arpit Agarwal
5f7926dcf0
Fix CHANGES.txt for HDFS-7722.
2015-03-18 12:53:02 -07:00
Arpit Agarwal
2b589da779
HDFS-7951. Fix NPE for TestFsDatasetImpl#testAddVolumeFailureReleasesInUseLock on Linux. (Contributed by Xiaoyu Yao)
2015-03-18 12:46:17 -07:00
Arpit Agarwal
ae6d273d63
HDFS-7950. Fix TestFsDatasetImpl#testAddVolumes failure on Windows. (Contributed by Xiaoyu Yao)
2015-03-18 12:35:41 -07:00
Arpit Agarwal
8944bb3d2e
HDFS-7948. TestDataNodeHotSwapVolumes#testAddVolumeFailures failed on Windows. (Contributed by Xiaoyu Yao)
2015-03-18 12:26:22 -07:00
Haohui Mai
5293ae0311
HDFS-7953. NN Web UI fails to navigate to paths that contain #. Contributed by kanaka kumar avvaru.
2015-03-18 11:08:49 -07:00
Arpit Agarwal
26c35438f3
HDFS-7946. TestDataNodeVolumeFailureReporting NPE on Windows. (Contributed by Xiaoyu Yao)
2015-03-17 21:30:13 -07:00
Colin Patrick Mccabe
d2dad74421
HDFS-7940. Add tracing to DFSClient#setQuotaByStorageType (Rakesh R via Colin P. McCabe)
...
(cherry picked from commit d8846707c5
)
(cherry picked from commit 455d4aa8a1
)
2015-03-17 11:02:05 -07:00
yliu
ef9d46dcb6
HDFS-7838. Expose truncate API for libhdfs. (yliu)
2015-03-17 07:25:58 +08:00
Colin Patrick Mccabe
861cc05092
HDFS-7722. DataNode#checkDiskError should also remove Storage when error is found. (Lei Xu via Colin P. McCabe)
...
(cherry picked from commit b49c3a1813
)
(cherry picked from commit 7455412a24
)
2015-03-16 18:42:21 -07:00
Colin Patrick Mccabe
647d9ae27a
HDFS-7915. The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error (cmccabe)
...
(cherry picked from commit bc9cb3e271
)
(cherry picked from commit c71b54fa32
)
2015-03-16 18:39:58 -07:00
Konstantin V Shvachko
ad4d704fbb
HDFS-7886. Fix TestFileTruncate falures. Contributed by Plamen Jeliazkov and Konstantin Shvachko.
2015-03-16 12:43:46 -07:00
Konstantin V Shvachko
41530b4173
HDFS-2605. Remove redundant "Release 0.21.1" section from CHANGES.txt. Contributed by Allen Wittenauer.
2015-03-13 13:45:47 -07:00
Konstantin V Shvachko
f6bdcd938e
HDFS-7903. Cannot recover block after truncate and delete snapshot. Contributed by Plamen Jeliazkov.
2015-03-13 12:45:47 -07:00
Kihwal Lee
116a7f1a16
HDFS-7435. PB encoding of block reports is very inefficient. Contributed by Daryn Sharp.
...
(cherry picked from commit d324164a51
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
(cherry picked from commit 464271a5ed
)
2015-03-13 14:42:02 -05:00
Brandon Li
cdeb1079ea
HDFS-7926. NameNode implementation of ClientProtocol.truncate(..) is not idempotent. Contributed by Tsz Wo Nicholas Sze
...
(cherry picked from commit f446669afb
)
(cherry picked from commit af80a98ace
)
2015-03-13 10:47:25 -07:00
Tsz-Wo Nicholas Sze
a1fffc3fcc
HDFS-6833. DirectoryScanner should not register a deleting block with memory of DataNode. Contributed by Shinichi Yamashita
2015-03-13 02:29:24 +08:00
Akira Ajisaka
718cd0f1f4
HDFS-7880. Remove the tests for legacy Web UI in branch-2. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit 6c80a3d34a
)
2015-03-12 10:55:47 +09:00
Colin Patrick Mccabe
8241228db7
HDFS-7830. DataNode does not release the volume lock when adding a volume fails. (Lei Xu via Colin P. McCabe)
...
(cherry picked from commit 5c1036d598
)
(cherry picked from commit eefca23e8c
)
2015-03-12 10:49:48 +09:00
Haohui Mai
245f7b2a77
HADOOP-11633. Convert remaining branch-2 .apt.vm files to markdown. Contributed by Masatake Iwasaki.
2015-03-11 14:31:16 -07:00
cnauroth
0fe5f5b5b8
HDFS-7491. Add incremental blockreport latency to DN metrics. Contributed by Ming Ma.
...
(cherry picked from commit fb34f45727
)
(cherry picked from commit 5e99b7cd2f
)
2015-03-11 14:12:06 -07:00
Jing Zhao
368ab2cd37
HDFS-7898. Change TestAppendSnapshotTruncate to fail-fast. Contributed by Tsz Wo Nicholas Sze.
...
(cherry picked from commit e43882e84a
)
(cherry picked from commit c7105fcff0
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-03-10 10:31:41 -07:00
Akira Ajisaka
c9a2e6f5db
HDFS-6806. HDFS Rolling upgrade document should mention the versions available. Contributed by J.Andreina.
...
(cherry picked from commit 82db3341bf
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-03-10 11:27:30 +09:00
Tsuyoshi Ozawa
28f4e6b222
HADOOP-11602. Backport 'Fix toUpperCase/toLowerCase to use Locale.ENGLISH.' (ozawa)
...
(cherry picked from commit b46f9e72db
)
2015-03-09 19:54:45 +09:00
Chris Douglas
3bc4f3502a
HDFS-7411. Change decommission logic to throttle by blocks rather
...
than nodes in each interval. Contributed by Andrew Wang
2015-03-08 18:32:20 -07:00
Brandon Li
c8af3b2768
HDFS-6488. Support HDFS superuser in NFSv3 gateway. Contributed by Brandon Li
...
(cherry picked from commit 0f8ecb1d0c
)
2015-03-06 15:22:20 -08:00
Haohui Mai
ab251fd355
HDFS-7818. OffsetParam should return the default value instead of throwing NPE when the value is unspecified. Contributed by Eric Payne.
2015-03-06 14:26:47 -08:00
Jing Zhao
994dadb9ba
HDFS-7885. Datanode should not trust the generation stamp provided by client. Contributed by Tsz Wo Nicholas Sze.
...
(cherry picked from commit 24db0812be
)
2015-03-06 10:57:38 -08:00
Jing Zhao
c5d52c7b52
HDFS-7855. Separate class Packet from DFSOutputStream. Contributed by Li Bo.
...
(cherry picked from commit 952640fa4c
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
2015-03-05 11:11:08 -08:00
Tsuyoshi Ozawa
f805d48b19
HADOOP-11648. Set DomainSocketWatcher thread name explicitly. Contributed by Liang Xie.
...
(cherry picked from commit 74a4754d1c
)
2015-03-05 16:06:37 +09:00
Tsz-Wo Nicholas Sze
f9a2007aff
HDFS-7746. Add a test randomly mixing append, truncate and snapshot operations.
2015-03-05 10:23:14 +08:00
Dongming Liang
ada545d3b7
HDFS-1522. Combine two BLOCK_FILE_PREFIX constants into one. Contributed by Dongming Liang.
2015-03-04 17:52:25 -08:00
Kihwal Lee
f85530f649
HDFS-7434. DatanodeID hashCode should not be mutable. Contributed by Daryn Sharp.
...
(cherry picked from commit 722b479469
)
2015-03-04 17:23:00 -06:00
Jing Zhao
b2ccf54c14
HDFS-7535. Utilize Snapshot diff report for distcp. Contributed by Jing Zhao.
...
(cherry picked from commit ed70fa142c
)
2015-03-04 10:32:39 -08:00
Haohui Mai
d020544ef6
HDFS-7879. hdfs.dll does not export functions of the public libhdfs API. Contributed by Chris Nauroth.
2015-03-04 09:17:29 -08:00
Vinayakumar B
c24448c27b
HDFS-7869. Inconsistency in the return information while performing rolling upgrade ( Contributed by J.Andreina )
...
(cherry picked from commit 3560180b6e
)
2015-03-04 14:39:51 +05:30
Akira Ajisaka
31b3f84601
HADOOP-11666. Revert the format change of du output introduced by HADOOP-6857. Contributed by Byron Wong.
2015-03-03 21:46:28 -08:00
Aaron T. Myers
bce3d442ff
HDFS-7682. {{DistributedFileSystem#getFileChecksum}} of a snapshotted file includes non-snapshotted content. Contributed by Charles Lamb.
...
(cherry picked from commit f2d7a67a2c
)
2015-03-03 18:09:31 -08:00
Haohui Mai
65bfde552a
HDFS-6565. Use jackson instead jetty json in hdfs-client. Contributed by Akira AJISAKA.
2015-03-03 17:58:21 -08:00
Arpit Agarwal
7e6624c218
HDFS-7757. Misleading error messages in FSImage.java. (Contributed by Brahma Reddy Battula)
2015-03-03 10:55:32 -08:00
Jing Zhao
6090f51725
HDFS-7871. NameNodeEditLogRoller can keep printing 'Swallowing exception' message. Contributed by Jing Zhao.
...
(cherry picked from commit b442aeec95
)
2015-03-02 20:23:10 -08:00
Haohui Mai
651aa8761e
HDFS-7785. Improve diagnostics information for HttpPutFailedException. Contributed by Chengbing Liu.
2015-03-02 15:35:19 -08:00
Gera Shegalov
52d0724a20
HDFS-7789. DFSck should resolve the path to support cross-FS symlinks. (gera)
...
(cherry picked from commit cbb492578e
)
2015-03-02 01:05:15 -08:00
Tsz-Wo Nicholas Sze
a5f3156b30
HDFS-7439. Add BlockOpResponseProto's message to the exception messages. Contributed by Takanobu Asanuma
2015-03-02 15:05:06 +08:00
Akira Ajisaka
d664705fd1
HDFS-5853. Add "hadoop.user.group.metrics.percentiles.intervals" to hdfs-default.xml (aajisaka)
...
(cherry picked from commit aa55fd3096
)
2015-03-01 01:18:36 -08:00
Tsz-Wo Nicholas Sze
c52636df3f
recommit "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir."
...
(cherry picked from commit acc172e371
)
2015-02-27 14:20:26 -08:00
Akira Ajisaka
0f9289e848
HDFS-7685. Document dfs.namenode.heartbeat.recheck-interval in hdfs-default.xml. Contributed by Kai Sasaki.
...
(cherry picked from commit 8719cdd4f6
)
2015-02-27 12:18:46 -08:00
Tsz-Wo Nicholas Sze
657b027bb2
HDFS-7308. Change the packet chunk size computation in DFSOutputStream in order to enforce packet size <= 64kB. Contributed by Takuya Fukudome
2015-02-27 23:46:42 +08:00
Vinayakumar B
bc60404eaf
HDFS-6753. Initialize checkDisk when DirectoryScanner not able to get files list for scanning (Contributed by J.Andreina)
...
(cherry picked from commit 4f75b15628
)
2015-02-27 16:37:03 +05:30
Tsz-Wo Nicholas Sze
d223a4a594
Revert "HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir."
...
This reverts commit acc172e371
.
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-02-27 18:27:19 +08:00
cnauroth
5dfb793b05
HDFS-7774. Unresolved symbols error while compiling HDFS on Windows 7/32 bit. Contributed by Kiran Kumar M R.
...
(cherry picked from commit c6d5b37a1c
)
2015-02-26 13:37:58 -08:00
Colin Patrick Mccabe
8b3b9568b6
HDFS-7819. Log WARN message for the blocks which are not in Block ID based layout (Rakesh R via Colin P. McCabe)
...
(cherry picked from commit f0c980abed
)
2015-02-26 12:03:11 -08:00
Vinayakumar B
f0f3d1937d
HDFS-7832. Show 'Last Modified' in Namenode's 'Browse Filesystem' (Contributed by Vinayakumar B)
...
(cherry picked from commit 166eecf687
)
2015-02-26 14:37:03 +05:30
Tsz-Wo Nicholas Sze
22b13a0ddb
HDFS-7537. Add "UNDER MIN REPL'D BLOCKS" count to fsck. Contributed by GAO Rui
2015-02-26 11:47:04 +08:00
Tsz-Wo Nicholas Sze
12fe3afcd3
HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade.
2015-02-26 10:15:49 +08:00
Benoy Antony
4e400030f6
HDFS-7467. Provide storage tier information for a directory via fsck. (Benoy Antony)
2015-02-25 16:46:41 -08:00
Colin Patrick Mccabe
d389a1ae98
HDFS-7495. Remove updatePosition argument from DFSInputStream#getBlockAt() (cmccabe)
...
(cherry picked from commit caa42adf20
)
2015-02-25 13:39:02 -08:00
Colin Patrick Mccabe
5c0073e706
HDFS-7668. Backport "Convert site documentation from apt to markdown" to branch-2 (Masatake Iwasaki via Colin P. McCabe)
...
(backported from commit 2f1e5dc628
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/ExtendedAttributes.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/Federation.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithNFS.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSHighAvailabilityWithQJM.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsDesign.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsNfsGateway.apt.vm
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HdfsQuotaAdminGuide.apt.vm
2015-02-24 16:18:18 -08:00
Andrew Wang
efb7e287f4
HDFS-7763. fix zkfc hung issue due to not catching exception in a corner case. Contributed by Liang Xie.
...
(cherry picked from commit 7105ebaa9f
)
2015-02-24 15:31:18 -08:00
Jing Zhao
8346427929
HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks(). Contributed by Konstantin Shvachko.
...
(cherry picked from commit 73bcfa99af
)
2015-02-24 10:34:42 -08:00
Tsuyoshi Ozawa
280dda5bcd
HDFS-7008. xlator should be closed upon exit from DFSAdmin#genericRefresh(). (ozawa)
...
(cherry picked from commit b53fd7163b
)
2015-02-24 23:59:59 +09:00
Colin Patrick Mccabe
453b210d0a
HDFS-7805. NameNode recovery prompt should be printed on console (Surendra Singh Lilhore via Colin P. McCabe)
...
(cherry picked from commit faaddb6ecb
)
2015-02-23 19:06:01 -08:00
Colin Patrick Mccabe
f4da6f7193
HDFS-7807. libhdfs htable.c: fix htable resizing, add unit test (cmccabe)
...
(cherry picked from commit 585768667e
)
2015-02-23 18:45:44 -08:00
cnauroth
657a6e389b
HDFS-7009. Active NN and standby NN have different live nodes. Contributed by Ming Ma.
...
(cherry picked from commit 769507bd7a
)
2015-02-23 15:12:36 -08:00
Arpit Agarwal
83f779e49a
HDFS-7806. Refactor: move StorageType from hadoop-hdfs to hadoop-common. (Contributed by Xiaoyu Yao)
2015-02-21 15:48:59 -08:00
yliu
eca1588db8
HDFS-7740. Test truncate with DataNodes restarting. (yliu)
2015-02-21 06:33:58 +08:00
cnauroth
9f06fb3aef
HDFS-7813. TestDFSHAAdminMiniCluster#testFencer testcase is failing frequently. Contributed by Rakesh R.
...
(cherry picked from commit 0d6af574e0
)
2015-02-20 17:01:22 -08:00
cnauroth
e09ba94f28
HDFS-7773. Additional metrics in HDFS to be accessed via jmx. Contributed by Anu Engineer.
2015-02-20 12:38:34 -08:00
cnauroth
ad77dfc3e4
HDFS-7814. Fix usage string of storageType parameter for "dfsadmin -setSpaceQuota/clrSpaceQuota". Contributed by Xiaoyu Yao.
...
(cherry picked from commit 8c6ae0d619
)
2015-02-20 12:22:00 -08:00
Kihwal Lee
b9157f92fc
HDFS-7788. Post-2.6 namenode may not start up with an image containing inodes created with an old release. Contributed by Rushabh Shah.
...
(cherry picked from commit 7ae5255a16
)
2015-02-20 09:09:56 -06:00
Harsh J
5c2c6b00dd
HDFS-7752. Improve description for "dfs.namenode.num.extra.edits.retained" and "dfs.namenode.num.checkpoints.retained" properties on hdfs-default.xml. Contributed by Wellington Chevreuil.
...
(cherry picked from commit b9a17909ba
)
2015-02-20 19:21:34 +05:30
yliu
fee29e4a4f
HDFS-7656. Expose truncate API for HDFS httpfs. (yliu)
2015-02-19 08:38:25 +08:00
Haohui Mai
3605928bd0
HDFS-7808. Remove obsolete -ns options in in DFSHAAdmin.java. Contributed by Arshad Mohammad.
2015-02-18 15:14:53 -08:00
cnauroth
b1b1e05e18
HDFS-7772. Document hdfs balancer -exclude/-include option in HDFSCommands.html. Contributed by Xiaoyu Yao.
2015-02-18 12:03:00 -08:00
Akira Ajisaka
6d6c684567
HDFS-7780. Update use of Iterator to Iterable in DataXceiverServer and SnapshotDiffInfo. Contributed by Ray Chiang.
...
(cherry picked from commit 6dc8812a95
)
2015-02-17 14:49:03 -08:00
Haohui Mai
f92a4904bf
HDFS-6662. WebHDFS cannot open a file if its path contains "%". Contributed by Gerson Carlos.
2015-02-17 13:05:00 -08:00
Kihwal Lee
8b37b4a78b
HDFS-7795. Show warning if not all favored nodes were chosen by namenode. Contributed by Kihwal Lee.
...
(cherry picked from commit db6606223c
)
2015-02-17 13:06:41 -06:00
Uma Maheswara Rao G
2cbac36fd3
HDFS-4266. BKJM: Separate write and ack quorum (Rakesh R via umamahesh)
...
(cherry picked from commit f0412de1c1
)
2015-02-17 21:31:43 +05:30
Uma Maheswara Rao G
35fecb5306
HDFS-7797. Add audit log for setQuota operation (Rakesh R via umamahesh)
...
(cherry picked from commit f24a56787a
)
2015-02-17 20:16:34 +05:30
Tsuyoshi Ozawa
00fb0710b6
HADOOP-11600. Fix up source codes to be compiled with Guava 17.0. (ozawa)
...
(cherry picked from commit 2f0f756b26
)
2015-02-17 21:57:58 +09:00
Vinayakumar B
91a5d92916
HDFS-7703. Support favouredNodes for the append for new blocks ( Contributed by Vinayakumar B)
...
(cherry picked from commit 89a5449280
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-02-17 15:19:03 +05:30
yliu
54e33baaf6
HDFS-7798. Checkpointing failure caused by shared KerberosAuthenticator. (Chengbing Liu via yliu)
2015-02-17 07:50:14 +08:00
cnauroth
441dfa4867
HDFS-7604. Track and display failed DataNode storage locations in NameNode. Contributed by Chris Nauroth.
...
(cherry picked from commit 9729b244de
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/HeartbeatManager.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
2015-02-16 14:44:49 -08:00
Arpit Agarwal
c6aa6cbfe5
HDFS-7775. Use consistent naming for NN-internal quota related types and functions. (Contributed bu Xiaoyu Yao)
2015-02-13 21:01:54 -08:00
cnauroth
751b834ad1
HDFS-7778. Rename FsVolumeListTest to TestFsVolumeList and commit it to branch-2. Contributed by Lei (Eddy) Xu.
...
(cherry picked from commit 2efb2347a9
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestFsVolumeList.java
2015-02-13 16:30:53 -08:00
Akira Ajisaka
3ad1d8d533
HDFS-4265. BKJM doesn't take advantage of speculative reads. Contributed by Rakesh R.
...
(cherry picked from commit 0d521e3326
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-02-13 15:25:13 -08:00
Colin Patrick Mccabe
7b9c978df2
HDFS-7686. Re-add rapid rescan of possibly corrupt block feature to the block scanner (cmccabe)
...
(cherry picked from commit 8bb9a5000e
)
2015-02-13 15:19:04 -08:00
Colin Patrick Mccabe
6edcb9a503
HDFS-7721. The HDFS BlockScanner may run fast during the first hour (cmccabe)
...
(cherry picked from commit 115428176e
)
2015-02-13 15:18:10 -08:00
Colin Patrick Mccabe
a54b4a7fc6
HDFS-7430 backport addendum. Add missing change. (cmccabe)
2015-02-13 15:11:14 -08:00
Colin Patrick Mccabe
a8c6a96f3b
HDFS-7430. Refactor the BlockScanner to use O(1) memory and use multiple threads (cmccabe)
...
(cherry picked from commit 6e62a1a672
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsDatasetSpi.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/FsVolumeSpi.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsVolumeListTest.java
2015-02-13 15:04:06 -08:00
Arpit Agarwal
f507791554
HDFS-7776. Adding additional unit tests for Quota By Storage Type. (Contributed by Xiaoyu Yao)
2015-02-13 12:02:54 -08:00
Allen Wittenauer
730597c20c
HDFS-316. Balancer should run for a configurable # of iterations (Xiaoyu Yao via aw)
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
hadoop-hdfs-project/hadoop-hdfs/src/site/apt/HDFSCommands.apt.vm
2015-02-12 18:02:23 -08:00
Colin Patrick Mccabe
a9a361c549
HDFS-7790. Do not create optional fields in DFSInputStream unless they are needed (cmccabe)
...
(cherry picked from commit 871cb56152
)
2015-02-12 17:49:11 -08:00
Akira Ajisaka
5696376e29
HDFS-7684. The host:port settings of the deamons should be trimmed before use. Contributed by Anu Engineer.
...
(cherry picked from commit 9235e2fb4e94a86671e1494f293c973cf572281c)
2015-02-12 17:40:48 -08:00
Arpit Agarwal
676fc2d1dc
HDFS-7723. Quota By Storage Type namenode implemenation. (Contributed by Xiaoyu Yao)
2015-02-12 10:50:14 -08:00
Colin Patrick Mccabe
e35788aa5a
HDFS-7694. FSDataInputStream should support "unbuffer" (cmccabe)
...
(cherry picked from commit 6b39ad0865
)
2015-02-12 10:42:14 -08:00
Jing Zhao
2156e38d5e
HDFS-3689. Add support for variable length block. Contributed by Jing Zhao.
...
(cherry picked from commit 2848db814a
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirConcatOp.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
2015-02-12 10:23:13 -08:00
Kihwal Lee
146f559097
Adding missing files from HDFS-7404
...
(cherry picked from commit 8d7215d40f
)
2015-02-12 11:18:41 -06:00
Kihwal Lee
39594301e1
HDFS-7704. DN heartbeat to Active NN may be blocked and expire if connection to Standby NN continues to time out. Contributed by Rushabh Shah.
...
(cherry picked from commit 38262779bb
)
2015-02-12 09:17:10 -06:00
yliu
f8d0825f39
HDFS-7761. cleanup unnecssary code logic in LocatedBlock. (yliu)
2015-02-12 03:09:38 +08:00
Tsz-Wo Nicholas Sze
65a6cf47ec
HDFS-6133. Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover. Contributed by zhaoyunjiong
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
2015-02-11 15:16:34 -08:00
Colin Patrick Mccabe
2ca76df21a
HDFS-7771. fuse_dfs should permit FILE: on the front of KRB5CCNAME (cmccabe)
...
(cherry picked from commit 50625e660a
)
2015-02-11 12:14:56 -08:00
Haohui Mai
744db47872
HDFS-7736. Fix typos in dfsadmin/fsck/snapshotDiff usage messages. Contributed by Brahma Reddy Battula.
2015-02-11 11:24:44 -08:00
Tsz-Wo Nicholas Sze
b49eeddbaf
HDFS-7751. Fix TestHDFSCLI for quota with storage type. Contributed by Xiaoyu Yao
2015-02-11 11:10:05 -08:00
Arpit Agarwal
8fa0fc9574
HDFS-7720. Quota by Storage Type API, tools and ClientNameNode Protocol changes. (Contributed by Xiaoyu Yao)
2015-02-11 11:07:53 -08:00
Konstantin V Shvachko
04789db74c
HDFS-7760. Document truncate for WebHDFS. Contributed by Konstantin Shvachko.
2015-02-11 10:05:53 -08:00
Konstantin V Shvachko
7f18176cd7
HDFS-7753. Fix Multithreaded correctness Warnings in BackupImage. Contributed by Rakesh R and Konstantin Shvachko.
2015-02-11 00:52:12 -08:00
Tsz-Wo Nicholas Sze
acc172e371
HDFS-7769. TestHDFSCLI should not create files in hdfs project root dir.
2015-02-10 17:50:00 -08:00
Konstantin V Shvachko
2b722b904e
HDFS-7058. Tests for truncate CLI. Contributed by Dasha Boudnik.
2015-02-09 21:31:02 -08:00
Vinayakumar B
a1bf7aecf7
HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode.(Contributed by Vinayakumar B)
...
(cherry picked from commit 3d15728ff5
)
2015-02-10 10:45:14 +05:30
Colin Patrick Mccabe
643a8892d1
HDFS-7718. Store KeyProvider in ClientContext to avoid leaking key provider threads when using FileContext (Arun Suresh via Colin P. McCabe)
...
(cherry picked from commit 02340a24f2
)
2015-02-09 20:25:31 -08:00
Colin Patrick Mccabe
1465393d14
HDFS-7744. Fix potential NPE in DFSInputStream after setDropBehind or setReadahead is called (cmccabe)
...
(cherry picked from commit a9dc5cd706
)
2015-02-09 20:19:48 -08:00
yliu
cc0668ebe6
HDFS-7756. Restore method signature for LocatedBlock#getLocations(). (Ted Yu via yliu)
2015-02-10 02:00:32 +08:00
yliu
ae316705bb
HADOOP-11510. Expose truncate API via FileContext. (yliu)
2015-02-10 01:43:08 +08:00
Arpit Agarwal
ff900eb64a
HDFS-7647. DatanodeManager.sortLocatedBlocks sorts DatanodeInfos but not StorageIDs. (Contributed by Milan Desai)
2015-02-09 12:17:53 -08:00
Jing Zhao
1e49b40f1f
HDFS-7743. Code cleanup of BlockInfo and rename BlockInfo to BlockInfoContiguous. Contributed by Jing Zhao.
...
(cherry picked from commit 1382ae525c
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockCollection.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicy.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/CreateEditsLog.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestEditLog.java
2015-02-08 12:16:09 -08:00
Tsz-Wo Nicholas Sze
c5f18ba65b
HDFS-7738. Revise the exception message for recover lease; add more truncate tests such as truncate with HA setup, negative tests, truncate with other operations and multiple truncates.
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestHAAppend.java
2015-02-07 15:26:06 -08:00
Akira Ajisaka
964a3b0c3e
HDFS-7710. Remove dead code in BackupImage.java. Contributed by Xiaoyu Yao.
...
(cherry picked from commit cfb829ecd5
)
2015-02-06 19:00:58 -08:00
Akira Ajisaka
2ec796ba80
HDFS-7732. Fix the order of the parameters in DFSConfigKeys. Contributed by Brahma Reddy Battula.
...
(cherry picked from commit eaab959a3e
)
2015-02-06 11:28:18 -08:00
yliu
e2803f5a51
HDFS-7741. Remove unnecessary synchronized in FSDataInputStream and HdfsDataInputStream. (yliu)
2015-02-06 06:29:52 +08:00
yliu
44d800b353
HDFS-7698. Fix locking on HDFS read statistics and add a method for clearing them. (Colin P. McCabe via yliu)
2015-02-05 23:57:36 +08:00
yliu
3ffe5a1ed2
HDFS-7655. Expose truncate API for Web HDFS. (yliu)
2015-02-05 23:48:55 +08:00
Haohui Mai
67002a5fc0
HDFS-7270. Add congestion signaling capability to DataNode write protocol. Contributed by Haohui Mai.
2015-02-05 11:03:43 -08:00
Tsuyoshi Ozawa
970fdc3ad9
HDFS-7709. Fix findbug warnings in httpfs. Contributed by Rakesh R.
...
(cherry picked from commit 20660b7a67
)
2015-02-05 19:57:40 +09:00
Arpit Agarwal
5807afed0a
HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure. (Contributed by Arpit Agarwal)
2015-02-04 16:25:40 -08:00
Colin Patrick Mccabe
76c0757c6e
HDFS-7719. BlockPoolSliceStorage#removeVolumes fails to remove some in-memory state associated with volumes. (Lei (Eddy) Xu via Colin P. McCabe)
...
(cherry picked from commit 40a415799b
)
2015-02-04 12:46:39 -08:00
Andrew Wang
696266e99c
HDFS-7734. Class cast exception in NameNode#main. Contributed by Yi Liu.
...
(cherry picked from commit 9175105eea
)
2015-02-04 11:31:49 -08:00
Tsuyoshi Ozawa
f92d4fa20f
HADOOP-11546. Checkstyle failing: Unable to instantiate DoubleCheckedLockingCheck. (ozawa)
...
(cherry picked from commit ba58da29f4
)
2015-02-05 00:51:40 +09:00
Andrew Wang
debd63f0b0
HDFS-7712. Switch blockStateChangeLog to use slf4j.
...
(cherry picked from commit 3ae38ec7df
)
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
2015-02-03 15:04:16 -08:00
Kihwal Lee
c428d303f6
HDFS-7707. Edit log corruption due to delayed block removal again. Contributed by Yongjun Zhang
...
(cherry picked from commit 843806d03a
)
2015-02-03 14:46:24 -06:00
Haohui Mai
7cb70d3447
HDFS-6651. Deletion failure can leak inodes permanently. Contributed by Jing Zhao.
2015-02-02 16:32:56 -08:00
Tsz-Wo Nicholas Sze
8426c7d806
HDFS-7696. In FsDatasetImpl, the getBlockInputStream(..) and getTmpInputStreams(..) methods may leak file descriptors.
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImpl.java
2015-02-02 13:42:15 -08:00
Tsz-Wo Nicholas Sze
a5568a276d
HDFS-7681. Change ReplicaInputStreams constructor to take InputStream(s) instead of FileDescriptor(s). Contributed by Joe Pallas
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-02-02 13:30:07 -08:00
Tsz-Wo Nicholas Sze
2ecfd018f4
HDFS-5782. Change BlockListAsLongs constructor to take Replica as parameter type instead of concrete classes Block and ReplicaInfo. Contributed by David Powell and Joe Pallas
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
2015-02-02 13:24:02 -08:00
Tsz-Wo Nicholas Sze
e43b1428b4
HDFS-5631. Change BlockMetadataHeader.readHeader(..), ChunkChecksum class and constructor to public; and fix FsDatasetSpi to use generic type instead of FsVolumeImpl. Contributed by David Powell and Joe Pallas
2015-02-02 12:59:22 -08:00
Colin Patrick Mccabe
4bf14876b3
HADOOP-11498 addendum. Fix extra "import" statements in branch-2.7 patch backport
2015-01-30 16:07:12 -08:00
Colin Patrick Mccabe
80381b62aa
HADOOP-11498. Bump the version of HTrace to 3.1.0-incubating (Masatake Iwasaki via Colin P. McCabe)
...
(cherry picked from commit 09ad9a868a
)
2015-01-30 16:02:36 -08:00
Andrew Wang
fcd23b61e3
HDFS-7706. Switch BlockManager logging to use slf4j.
...
(cherry picked from commit 951b3608a8
)
2015-01-30 11:36:02 -08:00
Colin Patrick Mccabe
e8d154baea
HADOOP-11403. Avoid using sys_errlist on Solaris, which lacks support for it (Malcolm Kavalsky via Colin P. McCabe)
...
(cherry picked from commit e36ef3b402
)
2015-01-29 15:48:59 -08:00
Kihwal Lee
3b030731a0
HDFS-7603. The background replication queue initialization may not let others run. Contributed by Kihwal Lee.
...
(cherry picked from commit 89b07490f8
)
2015-01-29 12:37:34 -06:00
yliu
dc7950ef18
HDFS-7423. various typos and message formatting fixes in nfs daemon and doc. (Charles Lamb via yliu)
2015-01-29 04:20:14 +08:00
Jing Zhao
e8300957a7
HDFS-7611. deleteSnapshot and delete of a file can leave orphaned blocks in the blocksMap on NameNode restart. Contributed by Jing Zhao and Byron Wong.
...
(cherry picked from commit d244574d03
)
2015-01-28 15:25:42 -08:00
Andrew Wang
55bb4eca73
HDFS-6673. Add delimited format support to PB OIV tool. Contributed by Eddy Xu.
...
(cherry picked from commit caf7298e49
)
2015-01-28 12:36:44 -08:00
Colin Patrick Mccabe
ad865b1cd0
HDFS-7675. Remove unused member DFSClient.spanReceiverHost (cmccabe)
...
(cherry picked from commit d12dd47f45
)
2015-01-27 20:52:53 -08:00
yliu
d483ba25d7
HDFS-7677. DistributedFileSystem#truncate should resolve symlinks. (yliu)
2015-01-27 23:47:52 +08:00
Allen Wittenauer
874e38d81f
HDFS-7566. Remove obsolete entries from hdfs-default.xml (Ray Chiang via aw)
2015-01-27 14:32:21 -08:00
Haohui Mai
b3a63fa8a2
HDFS-7683. Combine usages and percent stats in NameNode UI. Contributed by Vinayakumar B.
2015-01-27 11:09:42 -08:00
Steve Loughran
aa16173a0d
HDFS-49. MiniDFSCluster.stopDataNode will always shut down a node in the cluster if a matching name is not found. (stevel)
2015-01-27 11:10:46 +00:00
Kihwal Lee
c081fa6ae8
HDFS-7224. Allow reuse of NN connections via webhdfs. Contributed by Eric Payne
...
(cherry picked from commit 2b0fa20f69
)
2015-01-26 08:15:37 -06:00
Konstantin V Shvachko
24f0d56afb
HDFS-7676. Fix TestFileTruncate to avoid bug of HDFS-7611. Contributed by Konstantin Shvachko.
2015-01-24 18:07:56 -08:00
yliu
5f70b7ecf0
HDFS-7659. truncate should check negative value of the new length. Contributed by Yi Liu.
2015-01-24 17:28:36 -08:00
Konstantin V Shvachko
de66227a57
HADOOP-11490. Expose truncate API via FileSystem and shell command. Contributed by Milan Desai.
2015-01-24 17:28:34 -08:00
Konstantin V Shvachko
6b22170bee
HDFS-7643. Test case to ensure lazy persist files cannot be truncated. (Contributed by Yi Liu)
2015-01-24 17:28:32 -08:00
yliu
8adffbb2c8
HDFS-7634. Disallow truncation of Lazy persist files. (Contributed by Yi Liu)
2015-01-24 16:34:08 -08:00
yliu
0fdd09a895
HDFS-7638. Small fix and few refinements for FSN#truncate. (yliu)
2015-01-24 16:29:50 -08:00
Konstantin V Shvachko
a116e1fe9e
HDFS-7606. Fix potential NPE in INodeFile.getBlocks(). Contributed by Byron Wong.
2015-01-24 16:16:58 -08:00
Konstantin V Shvachko
00a7ebab22
HDFS-7056. Snapshot support for truncate. Contributed by Konstantin Shvachko and Plamen Jeliazkov.
2015-01-24 16:06:41 -08:00
Plamen Jeliazkov
6ff9bde714
HDFS-3107. Introduce truncate. Contributed by Plamen Jeliazkov.
2015-01-24 16:01:07 -08:00
Allen Wittenauer
6f89692dc9
HDFS-7644. minor typo in HttpFS doc (Charles Lamb via aw)
2015-01-23 13:47:11 -08:00
cnauroth
639c7b0ddc
HDFS-3519. Checkpoint upload may interfere with a concurrent saveNamespace. Contributed by Ming Ma.
2015-01-22 16:03:17 -08:00
Arpit Agarwal
dcd14db1a8
HDFS-7575. Fix CHANGES.txt
2015-01-22 14:37:43 -08:00
Arpit Agarwal
1d9d166c0b
HDFS-7575. Upgrade should generate a unique storage ID for each volume. (Contributed by Arpit Agarwal)
2015-01-22 14:12:56 -08:00
Kihwal Lee
95858db0c1
HDFS-7548. Corrupt block reporting delayed until datablock scanner thread detects it. Contributed by Rushabh Shah.
2015-01-21 14:42:45 -06:00
Colin Patrick Mccabe
5ec2b6caa9
HDFS-7496. Fix FsVolume removal race conditions on the DataNode by reference-counting the volume instances (Lei (Eddy) Xu via Colin P. McCabe)
...
(cherry-picked from commit a17584936c
)
2015-01-21 11:43:42 -08:00
Tsz-Wo Nicholas Sze
1dc5483054
HDFS-3443. Fix NPE when namenode transition to active during startup by adding checkNNStartup() in NameNodeRpcServer. Contributed by Vinayakumar B
...
Conflicts:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
2015-01-21 11:37:58 -08:00
Colin Patrick Mccabe
7779f38e68
HDFS-7610. Fix removal of dynamically added DN volumes (Lei (Eddy) Xu via Colin P. McCabe)
...
(cherry picked from commit a17584936c
)
2015-01-20 20:15:33 -08:00
yliu
7b5aea8175
HDFS-7641. Update archival storage user doc for list/set/get block storage policies. (yliu)
2015-01-21 02:04:03 +08:00
yliu
3f37354daa
HDFS-7623. Add htrace configuration properties to core-default.xml and update user doc about how to enable htrace. (yliu)
2015-01-21 00:57:51 +08:00
Haohui Mai
755731a6c1
HDFS-7640. print NFS Client in the NFS log. Contributed by Brandon Li.
2015-01-19 17:29:57 -08:00
Jing Zhao
5da33c5b5d
HDFS-7637. Fix the check condition for reserved path. Contributed by Yi Liu.
...
(cherry picked from commit e843a0a8ce
)
2015-01-19 10:29:02 -08:00
Haohui Mai
29551e19ef
HDFS-7573. Consolidate the implementation of delete() into a single class. Contributed by Haohui Mai.
2015-01-17 12:56:44 -08:00
cnauroth
8e765b8513
HDFS-7632. MiniDFSCluster configures DataNode data directories incorrectly if using more than 1 DataNode and more than 2 storage locations per DataNode. Contributed by Chris Nauroth.
...
(cherry picked from commit ec4389cf72
)
2015-01-16 10:52:13 -08:00
cnauroth
b3d03980c4
HDFS-7635. Remove TestCorruptFilesJsp from branch-2. Contributed by Chris Nauroth.
2015-01-16 10:40:38 -08:00
Arpit Agarwal
f72e2bfedd
HDFS-7591. hdfs classpath command should support same options as hadoop classpath. (Contributed by Varun Saxena)
2015-01-16 09:37:54 -08:00
yliu
bdbf13ac46
HDFS-7189. Add trace spans for DFSClient metadata operations. (Colin P. McCabe via yliu)
2015-01-16 00:21:38 +08:00
Kihwal Lee
6f792fc198
HDFS-7457. DatanodeID generates excessive garbage. Contributed by Daryn Sharp.
...
(cherry picked from commit 780a6bf145
)
2015-01-15 16:45:06 -06:00
Kihwal Lee
0090157fff
HDFS-7615. Remove longReadLock. Contributed by Kihwal Lee.
...
(cherry picked from commit 44eed6cbc9
)
2015-01-15 16:40:18 -06:00
Allen Wittenauer
c060d60a40
HADOOP-8989. hadoop fs -find feature (Jonathan Allen via aw)
2015-01-15 11:56:16 -08:00
Tsz-Wo Nicholas Sze
80f2c41ea4
HDFS-2219. Change fsck to support fully qualified paths so that a particular namenode in a federated cluster with multiple namenodes can be specified in the path parameter.
2015-01-14 11:32:38 -08:00
Colin Patrick Mccabe
0578df3baa
HDFS-7585. Get TestEnhancedByteBufferAccess working on CPU architectures with page sizes other than 4096 (Sam Liu via Colin P. McCabe)
...
(cherry picked from commit 446545c496
)
2015-01-14 11:14:16 -08:00
cnauroth
27eecaef10
HDFS-7570. SecondaryNameNode need twice memory when calling reloadFromImageFile. Contributed by zhaoyunjiong.
...
(cherry picked from commit 85aec75ce5
)
2015-01-13 16:10:40 -08:00
Jing Zhao
6e2eb9d5cf
HDFS-7600. Refine hdfs admin classes to reuse common code. Contributed by Jing Zhao.
2015-01-12 17:35:44 -08:00
Colin Patrick Mccabe
a6ebe7a76b
HDFS-5445. PacketReceiver populates the packetLen field in PacketHeader incorrectly (Jonathan Mace via Colin P. McCabe)
...
(cherry picked from commit f761bd8fe4
)
2015-01-12 17:11:34 -08:00
Colin Patrick Mccabe
dde774f2a4
HDFS-7598. Remove dependency on old version of Guava in TestDFSClientCache#testEviction (Sangjin Lee via Colin P. McCabe)
...
(cherry picked from commit b3ddd7ee39
)
2015-01-12 17:08:08 -08:00
Kihwal Lee
33534a0c9a
HDFS-7533. Datanode sometimes does not shutdown on receiving upgrade shutdown command. Contributed by Eric Payne.
...
(cherry picked from commit 6bbf9fdd04
)
2015-01-12 15:38:17 -06:00
Colin Patrick Mccabe
e4bb94b0b8
HDFS-7326. Add documentation for hdfs debug commands (Vijay Bhat via Colin P. McCabe)
...
(cherry picked from commit b78b4a1536
)
2015-01-12 12:17:37 -08:00
yliu
cce7ab4721
HDFS-7323. Move the get/setStoragePolicy commands out from dfsadmin. (jing9 via yliu)
2015-01-12 00:45:27 +08:00
cnauroth
75e4e55e12
HDFS-7596. NameNode should prune dead storages from storageMap. Contributed by Arpit Agarwal.
...
(cherry picked from commit ef3c3a832c
)
2015-01-10 09:18:41 -08:00
Jing Zhao
4b589e7cfa
HDFS-7182. JMX metrics aren't accessible when NN is busy. Contributed by Ming Ma.
2015-01-09 17:36:30 -08:00
cnauroth
9a7c763b3f
HADOOP-11464. Reinstate support for launching Hadoop processes on Windows using Cygwin. Contributed by Chris Nauroth.
2015-01-09 15:01:33 -08:00
Colin Patrick Mccabe
a663faf741
HADOOP-11470. Remove some uses of obsolete guava APIs from the hadoop codebase (Sangjin Lee via Colin P. McCabe)
...
(cherry picked from commit ae91b13a4b
)
2015-01-08 16:26:11 -08:00
cnauroth
f0acb7c2a2
HDFS-7579. Improve log reporting during block report rpc failure. Contributed by Charles Lamb.
...
(cherry picked from commit 7e2d9a3242
)
2015-01-08 15:13:05 -08:00
cnauroth
501246e24a
HDFS-7589. Break the dependency between libnative_mini_dfs and libhdfs. Contributed by Zhanwei Wang.
...
(cherry picked from commit 708b1aa074
)
2015-01-08 10:23:53 -08:00