8c4680852b
HDFS-11094. Send back HAState along with NamespaceInfo during a versionRequest as an optional parameter. Contributed by Eric Badger
2016-12-05 11:48:58 -08:00
1b5cceaffb
Revert "HDFS-11201. Spelling errors in the logging, help, assertions and exception messages. Contributed by Grant Sohn."
...
This reverts commit b9522e86a5
2016-12-05 10:54:43 -08:00
b9522e86a5
HDFS-11201. Spelling errors in the logging, help, assertions and exception messages. Contributed by Grant Sohn.
2016-12-05 10:48:25 -08:00
c7ff34f8dc
HDFS-11156. Add new op GETFILEBLOCKLOCATIONS to WebHDFS REST API. Contributed by Weiwei Yang
2016-12-02 11:10:13 -08:00
96c574927a
HDFS-8674. Improve performance of postponed block scans. Contributed by Daryn Sharp.
2016-12-01 12:11:27 -06:00
e0fa49234f
HDFS-11180. Intermittent deadlock in NameNode when failover happens.
2016-12-01 23:08:59 +09:00
7226a71b1f
HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers and Andrew Wang.
2016-11-30 15:58:31 -08:00
625df87c7b
HDFS-8678. Bring back the feature to view chunks of files in the HDFS file browser. Contributed by Ivo Udelsmann.
2016-11-30 09:12:15 -08:00
51e6c1cc3f
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-30 15:52:56 +08:00
cfd8076f81
Revert due to an error "HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen"
...
This reverts commit 5614f847b2ef2a5b70bd9a06edc4eba06174c6.
2016-11-30 15:44:52 +08:00
aeecfa24f4
HADOOP-10930. Refactor: Wrap Datanode IO related operations. Contributed by Xiaoyu Yao.
2016-11-29 20:52:36 -08:00
eaaa32950c
HDFS-11149. Support for parallel checking of FsVolumes.
2016-11-29 20:31:02 -08:00
b407d53195
Revert "HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers."
...
This reverts commit 09451252fa
2016-11-29 10:50:10 -08:00
09451252fa
HDFS-5517. Lower the default maximum number of blocks per file. Contributed by Aaron T. Myers.
2016-11-29 00:12:35 -08:00
47ca9e26fb
HADOOP-13838. KMSTokenRenewer should close providers (xiaochen via rkanter)
2016-11-28 18:08:09 -08:00
a2b1ff0257
HADOOP-13828. Implement getFileChecksum(path, length) for ViewFileSystem. Contributed by Manoj Govindassamy.
2016-11-28 11:54:43 -08:00
5d5614f847
HDFS-10994. Support an XOR policy XOR-2-1-64k in HDFS. Contributed by Sammi Chen
2016-11-28 14:34:44 +08:00
209e805430
HADOOP-13506. Redundant groupid warning in child projects. Contributed by Kai Sasaki.
2016-11-28 14:34:57 +09:00
636f7547b7
HDFS-11175. Document uppercase key names are not supported in TransparentEncryption.md. Contributed by Yiqun Lin.
2016-11-25 17:48:54 +09:00
de4894936a
HDFS-11368. Erasure Coding: Deprecate replication-related config keys. Contributed by Rakesh R.
2016-11-23 16:42:06 -08:00
dd98a8005a
HDFS-11058. Implement 'hadoop fs -df' command for ViewFileSystem. Contributed by Manoj Govindassamy.
2016-11-23 16:40:39 -08:00
3a09e5970d
HDFS-11145. Implement getTrashRoot() for ViewFileSystem. (Manoj Govindassamy via lei)
2016-11-22 13:52:09 -08:00
613b902b98
HDFS-11148. Update DataNode to use StorageLocationChecker at startup.
2016-11-22 10:50:25 -08:00
afcf8d38e7
HADOOP-13646. Remove outdated overview.html. Contributed By Brahma Reddy Battula.
2016-11-22 19:45:45 +05:30
49a09179e3
HDFS-10966. Enhance Dispatcher logic on deciding when to give up a source DataNode. Contributed by Mark Wagner and Zhe Zhang.
2016-11-21 10:13:24 -06:00
f922067748
HDFS-11101. TestDFSShell#testMoveWithTargetPortEmpty fails intermittently. Contributed By Brahma Reddy Battula.
2016-11-21 19:34:52 +05:30
c68dad18ab
HDFS-11144. TestFileCreationDelete#testFileCreationDeleteParent fails wind bind exception. Contributed By Brahma Reddy Battula.
2016-11-21 10:58:34 +05:30
d232625f73
HDFS-11113. Document dfs.client.read.striped configuration in hdfs-default.xml. Contributed by Rakesh R.
2016-11-21 14:05:15 +09:00
f6ffa11635
HDFS-11117. Refactor striped file tests to allow flexibly test erasure coding policy. Contributed by Sammi Chen
2016-11-18 16:08:59 +06:00
c0b1a44f6c
HADOOP-13660. Upgrade commons-configuration version. Contributed by Sean Mackrory.
2016-11-17 22:48:35 -06:00
b2d4b7b1b9
HDFS-11134. Fix bind exception threw in TestRenameWhileOpen. Contributed By Yiqun Lin.
2016-11-17 10:22:30 +05:30
04a024b683
HDFS-11147. Remove confusing log output in FsDatasetImpl#getInitialVolumeFailureInfos. Contributed by Chen Liang.
2016-11-16 14:47:34 -08:00
c90891e7b3
HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin
2016-11-16 13:02:10 -08:00
4fcea8a0c8
HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla.
2016-11-15 14:47:53 -06:00
ff0b99eafe
HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen.
2016-11-14 11:05:29 -08:00
6efb8c9c60
HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin
2016-11-14 20:39:39 +05:30
96f43921ba
HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu
2016-11-12 10:35:22 -08:00
2ee18fc15e
HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula
2016-11-11 22:12:27 -08:00
3d26717777
HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup.
...
This closes #155 .
2016-11-11 15:02:52 -08:00
4484b48498
HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang.
2016-11-11 14:31:23 -08:00
aa6010ccca
HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin
2016-11-11 12:29:47 -08:00
1ae57f0f75
HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru.
2016-11-11 20:08:15 +05:30
8848a8a76c
HDFS-11116. Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue. Contributed by Yiqun Lin.
2016-11-11 13:45:30 +09:00
ca68f9cb5b
HDFS-9337. Validate required params for WebHDFS requests (Contributed by Jagadesh Kiran N)
2016-11-10 16:51:33 +05:30
71adf44c3f
HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge.
2016-11-09 17:16:07 -08:00
c619e9b43f
HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang.
2016-11-09 09:16:50 -08:00
ed0bebabaa
HDFS-9482. Replace DatanodeInfo constructors with a builder pattern. Contributed by Brahma Reddy Battula.
2016-11-08 18:17:07 -08:00
62d8c17dfd
HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou
2016-11-08 17:04:30 -08:00
e1c6ef2efa
HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey.
2016-11-08 16:33:55 -08:00
3fff158587
HDFS-11114. Support for running async disk checks in DataNode.
...
This closes #153 .
2016-11-07 18:45:53 -08:00
049e7d27be
HDFS-10970. Update jackson from 1.9.13 to 2.x in hadoop-hdfs.
2016-11-07 11:16:31 +09:00
d8bab3dcb6
HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu.
2016-11-04 18:06:55 -07:00
0c0ab102ab
HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou
2016-11-04 14:34:49 -07:00
69dd5fa2d4
HDFS-11098. Datanode in tests cannot start in Windows after HDFS-10638 (Contributed by Vinayakumar B)
2016-11-04 08:31:43 +05:30
5cad93d5c7
HADOOP-11088. Quash unnecessary safemode WARN message during NameNode startup. Contributed by Yiqun Lin.
2016-11-03 15:31:07 -07:00
7534aee09a
HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang
2016-11-03 13:49:10 -07:00
b71907b2ae
HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin.
2016-11-03 10:27:52 -07:00
730cb0cff6
HDFS-10566. Submit plan request should throw exception if Datanode is in non-REGULAR status. Contributed by Xiaobing Zhou.
2016-11-02 17:46:56 -07:00
b59206190e
HDFS-10909. De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads. (Manoj Govindassamy via lei)
2016-11-02 16:45:25 -07:00
9b0c17f852
HDFS-11095. BlockManagerSafeMode should respect extension period default config value (30s). Contributed by Mingliang Liu
2016-11-02 14:58:24 -07:00
f2800f523d
Revert "HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover."
...
This reverts commit bd7f5911c7
2016-11-02 13:46:33 -07:00
cb5cc0dc53
HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu.
2016-11-02 10:35:47 +05:30
bd7f5911c7
HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover.
2016-11-01 16:48:45 -07:00
fa1512a740
HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou
2016-11-01 14:42:03 -07:00
34173a42d4
HDFS-10633. DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-11-01 11:14:03 -07:00
808f5d8985
HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin.
2016-11-01 21:32:52 +05:30
0c49f73a6c
HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu
2016-10-31 16:00:33 -07:00
a9d68d2e8e
HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru.
2016-10-31 12:39:30 -07:00
e9c7a97089
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R.
2016-10-31 00:22:51 +05:30
8a9388e5f6
HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger
2016-10-28 10:53:14 -07:00
8fa2195697
HDFS-10926. Update staled configuration properties related to erasure coding. Contributed by Sammi Chen
2016-10-28 12:37:30 +06:00
57187fdb93
HDFS-11064. Mention the default NN rpc ports in hdfs-default.xml. Contributed by Yiqun Lin.
2016-10-27 18:13:06 -07:00
5877f20f9c
HADOOP-10075. Update jetty dependency to version 9 (rkanter)
2016-10-27 16:09:00 -07:00
9e03ee5279
HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou
2016-10-27 16:00:27 -07:00
f3ac1f41b8
HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen.
2016-10-27 15:58:25 -07:00
ae48c496dc
HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee
2016-10-27 14:17:16 -05:00
db41965996
Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu"
...
This reverts commit ac35ee9393
2016-10-27 14:07:28 -05:00
ac35ee9393
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu
2016-10-27 20:20:56 +05:30
1cf6ec4ad4
HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su)
2016-10-27 16:44:00 +05:30
77f2684dd3
HDFS-10769. BlockIdManager.clear doesn't reset the counter for blockGroupIdGenerator (Contributed by Yiqun Lin and Rakesh R)
2016-10-27 14:16:03 +05:30
9c680d0b05
HDFS-11049. The description of dfs.block.replicator.classname is not clear. Contributed by Yiqun Lin.
2016-10-27 16:30:57 +09:00
a895c66255
HDFS-9929. Duplicate keys in NAMENODE_SPECIFIC_KEYS (Contributed by Akira Ajisaka)
2016-10-27 12:20:15 +05:30
9f32364d28
HDFS-11038. DiskBalancer: support running multiple commands in single test. Contributed by Xiaobing Zhou.
2016-10-26 19:39:02 -07:00
55e1fb8e32
HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger
2016-10-26 14:11:38 -07:00
f209e93566
HDFS-10638. Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode. (Virajith Jalaparti via lei)
2016-10-26 10:32:35 -07:00
e90af4a89b
HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee.
2016-10-26 09:16:13 -05:00
287effff93
HDFS-10935. TestFileChecksum fails in some cases. Contributed by Sammi Chen
2016-10-26 09:40:31 +06:00
9dce9011ad
HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang.
2016-10-25 19:50:56 -07:00
9a8a386458
HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou.
2016-10-25 11:24:03 -07:00
f6367c5f44
HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee.
2016-10-25 10:19:13 -07:00
dc3272bfe8
HDFS-11046. Duplicate '-' in the daemon log name.
2016-10-25 10:07:43 +09:00
9d175853b0
HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp.
2016-10-24 17:14:51 -05:00
b18f35f097
HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla.
2016-10-24 08:28:44 -05:00
25438526ed
HDFS-11033. Add documents for native raw erasure coder in XOR codes. Contributed by SammiChen.
2016-10-21 16:20:07 -07:00
df857f0d10
HDFS-10975. fsck -list-corruptfileblocks does not report corrupt EC files. Contributed by Takanobu Asanuma.
2016-10-21 15:43:12 -07:00
61e30cf83c
HDFS-8410. Add computation time metrics to datanode for ECWorker. Contributed by SammiChen.
2016-10-21 13:12:41 -07:00
ae8bccd509
HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du
2016-10-21 12:59:54 -07:00
f63cd78f60
HDFS-10730. Fix some failed tests due to BindException. Contributed by Yiqun Lin
2016-10-21 18:16:39 +05:30
d7d87deece
HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou
2016-10-20 19:51:48 -07:00
5e83a21cb6
HDFS-10976. Report erasure coding policy of EC files in Fsck. Contributed by Wei-Chiu Chuang.
2016-10-20 13:06:43 -07:00
6d2da38d16
HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin.
2016-10-20 10:56:39 -07:00
f872c6bc03
HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula
2016-10-20 21:47:17 +05:30
6fb6b651e8
HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin
2016-10-20 21:28:16 +05:30
5c0bffddc0
HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah.
2016-10-20 10:48:35 -05:00
4c73be135c
HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula
2016-10-20 20:45:45 +05:30
73504b1bdc
HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin
2016-10-20 10:41:18 +05:30
b4564103e4
HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru.
2016-10-19 17:20:07 -07:00
c5573e6a75
HDFS-11009. Add a tool to reconstruct block meta file from CLI.
2016-10-18 22:42:28 -07:00
c62ae7107f
HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru.
2016-10-18 14:05:43 -07:00
d26a1bb9d6
HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R
2016-10-18 14:51:08 +06:00
c023c74886
HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki
2016-10-18 12:02:53 +06:00
bedfec0c10
HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini.
2016-10-18 15:00:44 +09:00
b61fb267b9
HDFS-9390. Block management for maintenance states.
2016-10-17 17:45:41 -07:00
8fd4c37c45
HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang.
2016-10-17 14:21:36 -07:00
5f4ae85bd8
HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula
2016-10-15 22:28:33 +05:30
391ce535a7
HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi.
2016-10-14 18:13:54 -07:00
30bb1970cc
HDFS-10558. DiskBalancer: Print the full path to plan file. Contributed by Xiaobing Zhou.
2016-10-14 17:07:59 -07:00
76cc84e6d4
HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu
2016-10-14 14:29:02 -07:00
8c520a27cb
HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)
2016-10-14 13:41:59 -07:00
adb96e109f
HDFS-10827. When there are unrecoverable ec block groups, Namenode Web UI doesn't show the block names. Contributed by Takanobu Asanuma.
2016-10-14 13:21:53 -07:00
0007360c33
HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu.
2016-10-14 11:41:29 -07:00
8a9f6635a3
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-14 11:38:48 -05:00
8c721aa00a
Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin."
...
This reverts commit fdce515091
2016-10-13 13:23:28 -07:00
332a61fd74
HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula.
2016-10-13 13:52:49 -05:00
fdce515091
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-13 11:41:37 -07:00
b371c56365
HDFS-10949. DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacity. Contributed by Xiaobing Zhou.
2016-10-13 10:26:07 -07:00
1291254042
HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu
2016-10-13 22:05:00 +05:30
9454dc5e80
HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee.
2016-10-13 08:47:15 -05:00
901eca004d
HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu.
2016-10-13 14:29:30 +09:00
85cd06f663
HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah.
2016-10-12 15:11:42 -05:00
7ba7092bbc
HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou
2016-10-11 17:23:54 -07:00
3c9a01062e
HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang
2016-10-11 16:29:30 -07:00
61f0490a73
HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle.
2016-10-11 15:55:02 -07:00
809cfd27a3
HDFS-10916. Switch from "raw" to "system" xattr namespace for erasure coding policy. (Andrew Wang via lei)
2016-10-11 10:04:46 -07:00
96b12662ea
HDFS-10637. Modifications to remove the assumption that FsVolumes are backed by java.io.File. (Virajith Jalaparti via lei)
2016-10-10 15:30:03 -07:00
b963818621
HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula
2016-10-10 13:19:17 -07:00
3441c746b5
HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou
2016-10-10 11:33:37 -07:00
af50da3298
HDFS-10895. Update HDFS Erasure Coding doc to add how to use ISA-L based coder. Contributed by Sammi Chen
2016-10-10 11:55:49 +06:00
4d106213c0
HDFS-10968. BlockManager#isInNewRack should consider decommissioning nodes. Contributed by Jing Zhao.
2016-10-07 22:44:54 -07:00
6a38d118d8
HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory.
2016-10-07 17:37:15 -07:00
e57fa81d95
HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp.
2016-10-07 17:20:15 -05:00
3565c9af17
HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp.
2016-10-07 14:15:59 -05:00
3059b251d8
HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma.
2016-10-07 10:17:50 -07:00
be3cb10f53
HDFS-10969. Fix typos in hdfs-default.xml Contributed by Yiqun Lin
2016-10-07 22:18:40 +05:30
48b9d5fd2a
HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp.
2016-10-06 16:33:46 -05:00
1d330fbaf6
HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth.
2016-10-06 13:19:16 -07:00
f32e9fc8f7
HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn sharp.
2016-10-06 15:11:14 -05:00
311954883f
HDFS-10957. Retire BKJM from trunk (Vinayakumar B)
2016-10-06 19:28:25 +05:30
b90fc70d67
HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen
2016-10-06 17:47:31 +05:30
202325485d
HDFS-10893. Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test. Contributed by Mingliang Liu
2016-10-05 22:33:29 -07:00
c5ca216915
HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge.
2016-10-05 17:36:50 -07:00
e68c7b96c7
HDFS-10962. TestRequestHedgingProxyProvider is flaky.
2016-10-05 15:08:56 -07:00
8867762256
HDFS-10826. Correctly report missing EC blocks in FSCK. Contributed by Takanobu Asanuma.
2016-10-05 10:52:50 -07:00
44f48ee96e
HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp.
2016-10-04 15:05:09 -05:00
382307cbdd
HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah.
2016-10-04 11:09:03 -05:00
ef7f06f7d1
HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore
2016-10-04 18:46:42 +05:30
5ea0210c7f
HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N
2016-10-04 18:33:22 +05:30
f61e3d13e9
HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger
2016-10-03 22:34:41 -04:00
8078a5efd0
HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula
2016-10-03 21:50:16 -04:00
853d65a157
HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen.
2016-10-03 16:01:54 -07:00
607705c488
HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu.
2016-10-03 10:53:21 -07:00
744208431f
HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp.
2016-10-03 11:27:23 -05:00
90020624b0
HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp.
2016-10-03 09:13:04 -05:00
fe9ebe20ab
Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock."
...
This reverts commit c7ce6fdc20
2016-09-30 23:11:51 -07:00
c7ce6fdc20
HDFS-10923. Make InstrumentedLock require ReentrantLock.
2016-09-30 23:00:06 -07:00
434c5ea75d
HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen.
2016-09-30 13:16:08 -07:00
7fad1221d6
HDFS-10907. Fix Erasure Coding documentation. Contributed by Manoj Govindassamy.
2016-09-30 12:51:27 -07:00
ee33a02234
HDFS-10910. HDFS Erasure Coding doc should state its currently supported erasure coding policies. Contributed by Yiqun Lin.
2016-09-30 12:48:11 -07:00
2ab1ef15c5
HDFS-10908. Improve StripedBlockReader#createBlockReader error logging. Contributed by Manoj Govindassamy.
2016-09-30 12:35:47 -07:00
a0730aa5ce
HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp.
2016-09-30 13:03:24 -05:00
0670149c88
HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang.
2016-09-30 08:44:18 -05:00
84c6264079
HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu
2016-09-28 16:03:51 -07:00
92e5e91598
HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client.
2016-09-28 16:01:03 -07:00
5f34402ada
HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp.
2016-09-28 17:57:23 -05:00
c3b235e565
HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou.
2016-09-28 11:47:37 -07:00
03f519a757
HDFS-9850. DiskBalancer: Explore removing references to FsVolumeSpi. Contributed by Manoj Govindassamy.
2016-09-27 21:35:06 -07:00
6437ba18c5
HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou
2016-09-27 20:40:17 -07:00
9c9736463b
HDFS-10900. DiskBalancer: Complete the documents for the report command. Contributed by Yiqun Lin.
2016-09-27 19:28:41 -07:00
2acfb1e1e4
HDFS-10376. Enhance setOwner testing. (John Zhuge via Yongjun Zhang)
2016-09-27 14:55:28 -07:00
875062b5bc
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda)
2016-09-27 11:26:45 -07:00
8ae4729107
HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal)
2016-09-27 10:02:15 -07:00
f55eb981dd
HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin.
2016-09-28 01:29:24 +09:00
df1d0f5d56
HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula
2016-09-27 19:33:10 +05:30
edf0d0f8b2
HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki
2016-09-27 07:04:37 +05:30
059058f961
HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru.
2016-09-26 17:09:47 -07:00
8e06d865c4
Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications.
2016-09-26 14:41:04 -07:00
3ae652f821
HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang.
2016-09-26 13:11:32 -07:00
3e37e243ee
HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N
2016-09-25 11:02:46 +05:30
6eb700ee74
HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko.
2016-09-23 14:10:11 -07:00
74b3dd514c
HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy.
2016-09-23 13:26:57 -07:00
a5bb88c8e0
HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen.
2016-09-23 10:37:46 -07:00
bbdf350ff9
HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document.
2016-09-23 19:21:34 +05:30
c4480f7f42
HDFS-10894. Remove the redundant charactors for command -saveNamespace in HDFSCommands.md Contributed by Yiqun Lin.
2016-09-23 19:11:02 +05:30
d85d9b2e7b
HDFS-10871. DiskBalancerWorkItem should not import jackson relocated by htrace. Contributed by Manoj Govindassamy.
2016-09-22 19:36:16 -07:00
40acacee08
HDFS-10877. Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers. Contributed by Sean Mackrory.
2016-09-22 11:43:11 -07:00
964e546ab1
HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims)
2016-09-21 10:35:25 +09:00
0e918dff59
HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen.
2016-09-20 16:56:52 -07:00
734d54c1a8
HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen
2016-09-21 21:34:48 +08:00
e52d6e7a46
HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen.
2016-09-19 21:44:42 -07:00
c54f6ef30f
HDFS-10868. Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLED.
2016-09-19 11:17:03 -07:00
ea839bd48e
HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen.
2016-09-17 22:25:39 -07:00
b09a03cd7d
HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani.
2016-09-16 16:08:47 +09:00
ec3ea18875
HDFS-9895. Remove unnecessary conf cache from DataNode. Contributed by Xiaobing Zhou.
2016-09-15 15:25:47 -07:00
2a8f55a0cf
HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak.
2016-09-14 09:31:17 -07:00
ea0c2b8b05
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-15 10:46:00 +08:00
86c9862bec
HDFS-10636. Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. (Virajith Jalaparti via lei)
2016-09-13 12:54:14 -07:00
1c0d18f322
HDFS-10562. DiskBalancer: update documentation on how to report issues and debug. Contributed by Anu Engineer.
2016-09-13 12:42:10 -07:00
db6d243cf8
HDFS-10837. Standardize serializiation of WebHDFS DirectoryListing.
2016-09-13 11:02:36 -07:00
e3f7f58a5f
HDFS-10599. DiskBalancer: Execute CLI via Shell. Contributed by Manoj Govindassamy.
2016-09-13 09:38:12 -07:00
f0876b8b60
HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin.
2016-09-13 14:50:03 +09:00
d2466ec3e0
HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang)
2016-09-12 21:25:06 -07:00
72dfb048a9
HDFS-10858. FBR processing may generate incorrect reportedBlock-blockGroup mapping. Contributed by Jing Zhao.
2016-09-12 16:40:11 -07:00
8a93f45a80
HDFS-10821. DiskBalancer: Report command support with multiple nodes. Contributed by Yiqun Lin.
2016-09-12 15:45:26 -07:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
cc01ed7026
HDFS-10855. Fix typos in HDFS documents. Contributed by Yiqun Lin.
2016-09-10 23:24:49 -07:00
a99bf26a08
HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy)
2016-09-10 18:22:25 -07:00
bee9f57f5c
HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer.
2016-09-09 15:00:39 -07:00
a0b0383677
Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-09 10:54:14 -07:00
baab48922a
HDFS-9849. DiskBalancer: reduce lock path in shutdown code. Contributed by Yuanbo Liu.
2016-09-08 20:00:42 -07:00
35c5943b8b
HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy.
2016-09-08 19:26:56 -07:00
011f3b24d4
HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang
2016-09-08 17:53:40 -07:00
b6d839a60c
HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin
2016-09-08 13:29:30 -07:00
401db4fc65
HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng.
2016-09-08 11:54:33 -07:00
20a20c2f6e
HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin.
2016-09-09 01:34:34 +09:00
63f594892e
HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer.
2016-09-08 15:13:43 +09:00
d355573f56
Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng"
...
This reverts commit 62a9667136
2016-09-08 05:50:17 +08:00
5f23abfa30
HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
2016-09-06 13:37:21 -07:00
f6c0b7543f
HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee.
2016-09-06 11:48:35 -07:00
f0d5382ff3
HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth.
2016-09-06 11:02:39 -07:00
d37dc5d1b8
HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin
2016-09-06 10:38:04 -07:00
62a9667136
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-07 17:05:33 +08:00
07650bc37a
HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-02 15:33:11 -07:00
cbd909ce2a
HDFS-10833. Fix JSON errors in WebHDFS.md examples.
2016-09-02 14:50:34 -07:00
6c600360ca
Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2.
2016-08-31 23:43:59 -07:00
6f4b0d33ca
HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen.
2016-08-31 15:40:09 -07:00
85bab5fb57
HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator.
2016-08-31 14:29:37 -07:00
01721dd88e
HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang.
2016-08-31 14:02:37 -05:00
20ae1fa259
HDFS-10813. DiskBalancer: Add the getNodeList method in Command. Contributed by Yiqun Lin.
2016-08-30 18:42:55 -07:00
9dcbdbdb5a
HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma.
2016-08-30 14:00:13 -07:00
c4ee6915a1
HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan.
2016-08-30 10:43:20 -07:00
4bd45f54ee
HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi.
2016-08-29 23:37:26 -07:00
cd5e10ccca
HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge.
2016-08-29 17:48:08 -07:00
5d1609ddf2
HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin.
2016-08-29 13:59:54 -07:00
6742fb6e68
HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu)
2016-08-29 10:15:34 -07:00
c25817159a
HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang.
2016-08-27 22:51:31 -07:00
e806db7190
HDFS-10584. Allow long-running Mover tool to login with keytab. Contributed by Rakesh R.
2016-08-26 16:43:45 -07:00
a445b82baa
HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy.
2016-08-26 15:39:18 -07:00
407b519fb1
HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen.
2016-08-26 14:19:55 -07:00
8b7adf4ddf
HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp.
2016-08-26 15:39:56 -05:00
13fb1b50e6
HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki.
2016-08-26 09:53:10 -05:00
Mingliang Liu
Wei-Chiu Chuang
Kihwal Lee
Akira Ajisaka
Andrew Wang
Ravi Prakash
Kai Zheng
Xiaoyu Yao
Arpit Agarwal
Robert Kanter
Lei Xu
Brahma Reddy Battula
Zhe Zhang
Vinayakumar B
Xiao Chen
Anu Engineer
Robert Kanter
Erik Krogen
Chris Douglas
Ming Ma
Vinitha Reddy Gankidi
Jing Zhao
Chris Nauroth
Yongjun Zhang
Wangda Tan
Masatake Iwasaki
Konstantin V Shvachko
Allen Wittenauer