04a024b683
HDFS-11147. Remove confusing log output in FsDatasetImpl#getInitialVolumeFailureInfos. Contributed by Chen Liang.
2016-11-16 14:47:34 -08:00
c90891e7b3
HDFS-11105. TestRBWBlockInvalidation#testRWRInvalidation fails intermittently. Contributed by Yiqun Lin
2016-11-16 13:02:10 -08:00
4fcea8a0c8
HDFS-8870. Lease is leaked on write failure. Contributed by Kuhu Shukla.
2016-11-15 14:47:53 -06:00
5af572b644
HADOOP-13427. Eliminate needless uses of FileSystem#{exists(), isFile(), isDirectory()}. Contributed by Steve Loughran and Mingliang Liu
2016-11-15 10:57:00 -08:00
ff0b99eafe
HDFS-10872. Add MutableRate metrics for FSNamesystemLock operations. Contributed by Erik Krogen.
2016-11-14 11:05:29 -08:00
6efb8c9c60
HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed By Yiqun Lin
2016-11-14 20:39:39 +05:30
96f43921ba
HADOOP-13813. TestDelegationTokenFetcher#testDelegationTokenWithoutRenewer is failing. Contributed by Mingliang Liu
2016-11-12 10:35:22 -08:00
2ee18fc15e
HDFS-11129. TestAppendSnapshotTruncate fails with bind exception. Contributed by Brahma Reddy Battula
2016-11-11 22:12:27 -08:00
3d26717777
HDFS-11119. Support for parallel checking of StorageLocations on DataNode startup.
...
This closes #155 .
2016-11-11 15:02:52 -08:00
4484b48498
HDFS-10941. Improve BlockManager#processMisReplicatesAsync log. Contributed by Chen Liang.
2016-11-11 14:31:23 -08:00
aa6010ccca
HDFS-11122. TestDFSAdmin#testReportCommand fails due to timed out. Contributed by Yiqun Lin
2016-11-11 12:29:47 -08:00
1ae57f0f75
HDFS-11128. CreateEditsLog throws NullPointerException. Contributed by Hanisha Koneru.
2016-11-11 20:08:15 +05:30
8848a8a76c
HDFS-11116. Fix javac warnings caused by deprecation of APIs in TestViewFsDefaultValue. Contributed by Yiqun Lin.
2016-11-11 13:45:30 +09:00
ca68f9cb5b
HDFS-9337. Validate required params for WebHDFS requests (Contributed by Jagadesh Kiran N)
2016-11-10 16:51:33 +05:30
71adf44c3f
HDFS-11120. TestEncryptionZones should waitActive. Contributed by John Zhuge.
2016-11-09 17:16:07 -08:00
c619e9b43f
HDFS-11056. Concurrent append and read operations lead to checksum error. Contributed by Wei-Chiu Chuang.
2016-11-09 09:16:50 -08:00
ed0bebabaa
HDFS-9482. Replace DatanodeInfo constructors with a builder pattern. Contributed by Brahma Reddy Battula.
2016-11-08 18:17:07 -08:00
62d8c17dfd
HDFS-11083. Add unit test for DFSAdmin -report command. Contributed by Xiaobing Zhou
2016-11-08 17:04:30 -08:00
e1c6ef2efa
HADOOP-13789. Hadoop Common includes generated test protos in both jar and test-jar. Contributed by Sean Busbey.
2016-11-08 16:33:55 -08:00
3fff158587
HDFS-11114. Support for running async disk checks in DataNode.
...
This closes #153 .
2016-11-07 18:45:53 -08:00
049e7d27be
HDFS-10970. Update jackson from 1.9.13 to 2.x in hadoop-hdfs.
2016-11-07 11:16:31 +09:00
d8bab3dcb6
HDFS-10756. Expose getTrashRoot to HTTPFS and WebHDFS. Contributed by Yuanbo Liu.
2016-11-04 18:06:55 -07:00
0c0ab102ab
HDFS-11085. Add unit test for NameNode failing to start when name dir is unwritable. Contributed by Xiaobing Zhou
2016-11-04 14:34:49 -07:00
69dd5fa2d4
HDFS-11098. Datanode in tests cannot start in Windows after HDFS-10638 (Contributed by Vinayakumar B)
2016-11-04 08:31:43 +05:30
5cad93d5c7
HADOOP-11088. Quash unnecessary safemode WARN message during NameNode startup. Contributed by Yiqun Lin.
2016-11-03 15:31:07 -07:00
7534aee09a
HDFS-11076. Add unit test for extended Acls. Contributed by Chen Liang
2016-11-03 13:49:10 -07:00
b71907b2ae
HDFS-11097. Fix warnings for deprecated StorageReceivedDeletedBlocks constructor. Contributed by Yiqun Lin.
2016-11-03 10:27:52 -07:00
7e521c5a49
HDFS-11080. Update HttpFS to use ConfigRedactor. Contributed by Sean Mackrory.
2016-11-02 19:11:05 -07:00
730cb0cff6
HDFS-10566. Submit plan request should throw exception if Datanode is in non-REGULAR status. Contributed by Xiaobing Zhou.
2016-11-02 17:46:56 -07:00
b59206190e
HDFS-10909. De-duplicate code in ErasureCodingWorker#initializeStripedReadThreadPool and DFSClient#initThreadsNumForStripedReads. (Manoj Govindassamy via lei)
2016-11-02 16:45:25 -07:00
9b0c17f852
HDFS-11095. BlockManagerSafeMode should respect extension period default config value (30s). Contributed by Mingliang Liu
2016-11-02 14:58:24 -07:00
f2800f523d
Revert "HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover."
...
This reverts commit bd7f5911c7
2016-11-02 13:46:33 -07:00
cb5cc0dc53
HDFS-11031. Add additional unit test for DataNode startup behavior when volumes fail. Contributed by Mingliang Liu.
2016-11-02 10:35:47 +05:30
bd7f5911c7
HADOOP-13717. Normalize daemonization behavior of the diskbalancer with balancer and mover.
2016-11-01 16:48:45 -07:00
fa1512a740
HDFS-11065. Add space quota tests for heterogenous storages. Contributed by Xiaobing Zhou
2016-11-01 14:42:03 -07:00
34173a42d4
HDFS-10633. DiskBalancer : Add the description for the new setting dfs.disk.balancer.plan.threshold.percent in HDFSDiskbalancer.md. Contributed by Yiqun Lin.
2016-11-01 11:14:03 -07:00
808f5d8985
HDFS-10499. TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently. Contributed by Yiqun Lin.
2016-11-01 21:32:52 +05:30
0c49f73a6c
HDFS-11030. TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing). Contributed by Mingliang Liu
2016-10-31 16:00:33 -07:00
a9d68d2e8e
HDFS-11074. Remove unused method FsDatasetSpi#getFinalizedBlocksOnPersistentStorage. Contributed by Hanisha Koneru.
2016-10-31 12:39:30 -07:00
e9c7a97089
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Rakesh R.
2016-10-31 00:22:51 +05:30
8a9388e5f6
HDFS-11048. Audit Log should escape control characters. Contributed by Eric Badger
2016-10-28 10:53:14 -07:00
8fa2195697
HDFS-10926. Update staled configuration properties related to erasure coding. Contributed by Sammi Chen
2016-10-28 12:37:30 +06:00
57187fdb93
HDFS-11064. Mention the default NN rpc ports in hdfs-default.xml. Contributed by Yiqun Lin.
2016-10-27 18:13:06 -07:00
5877f20f9c
HADOOP-10075. Update jetty dependency to version 9 (rkanter)
2016-10-27 16:09:00 -07:00
9e03ee5279
HDFS-11047. Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode. Contributed by Xiaobing Zhou
2016-10-27 16:00:27 -07:00
f3ac1f41b8
HDFS-9500. Fix software version counts for DataNodes during rolling upgrade. Contributed by Erik Krogen.
2016-10-27 15:58:25 -07:00
31ff42b510
HDFS-11055. Update default-log4j.properties for httpfs to imporve test logging. Contributed by Wei-Chiu Chuang.
2016-10-27 13:37:00 -07:00
ae48c496dc
HDFS-11069. Tighten the authorization of datanode RPC. Contributed by Kihwal Lee
2016-10-27 14:17:16 -05:00
db41965996
Revert "HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu"
...
This reverts commit ac35ee9393
2016-10-27 14:07:28 -05:00
ac35ee9393
HDFS-10455. Logging the username when deny the setOwner operation. Contributed by Tianyin Xiu
2016-10-27 20:20:56 +05:30
1cf6ec4ad4
HDFS-8492. DN should notify NN when client requests a missing block (Contributed by Walter Su)
2016-10-27 16:44:00 +05:30
77f2684dd3
HDFS-10769. BlockIdManager.clear doesn't reset the counter for blockGroupIdGenerator (Contributed by Yiqun Lin and Rakesh R)
2016-10-27 14:16:03 +05:30
9c680d0b05
HDFS-11049. The description of dfs.block.replicator.classname is not clear. Contributed by Yiqun Lin.
2016-10-27 16:30:57 +09:00
a895c66255
HDFS-9929. Duplicate keys in NAMENODE_SPECIFIC_KEYS (Contributed by Akira Ajisaka)
2016-10-27 12:20:15 +05:30
0bdd263d82
HADOOP-13017. Implementations of InputStream.read(buffer, offset, bytes) to exit 0 if bytes==0. Contributed by Steve Loughran.
2016-10-27 15:46:59 +09:00
9f32364d28
HDFS-11038. DiskBalancer: support running multiple commands in single test. Contributed by Xiaobing Zhou.
2016-10-26 19:39:02 -07:00
55e1fb8e32
HDFS-10921. TestDiskspaceQuotaUpdate doesn't wait for NN to get out of safe mode. Contributed by Eric Badger
2016-10-26 14:11:38 -07:00
f209e93566
HDFS-10638. Modifications to remove the assumption that StorageLocation is associated with java.io.File in Datanode. (Virajith Jalaparti via lei)
2016-10-26 10:32:35 -07:00
e90af4a89b
HDFS-11053. Unnecessary superuser check in versionRequest(). Contributed by Kihwal Lee.
2016-10-26 09:16:13 -05:00
ce6bbfb23c
HDFS-11050. Change log level to 'warn' when ssl initialization fails and defaults to DEFAULT_TIMEOUT_CONN_CONFIGURATOR. Contributed by Kuhu Shukla.
2016-10-26 08:07:53 -05:00
287effff93
HDFS-10935. TestFileChecksum fails in some cases. Contributed by Sammi Chen
2016-10-26 09:40:31 +06:00
9dce9011ad
HDFS-11054. Suppress verbose log message in BlockPlacementPolicyDefault. Contributed by Chen Liang.
2016-10-25 19:50:56 -07:00
9a8a386458
HDFS-11011. Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota'. Contributed by Xiaobing Zhou.
2016-10-25 11:24:03 -07:00
f6367c5f44
HDFS-11015. Enforce timeout in balancer. Contributed by Kihwal Lee.
2016-10-25 10:19:13 -07:00
dc3272bfe8
HDFS-11046. Duplicate '-' in the daemon log name.
2016-10-25 10:07:43 +09:00
9d175853b0
HDFS-10997. Reduce number of path resolving methods. Contributed by Daryn Sharp.
2016-10-24 17:14:51 -05:00
b18f35f097
HDFS-11042. Add missing cleanupSSLConfig() call for tests that use setupSSLConfig(). Contributed by Kuhu Shukla.
2016-10-24 08:28:44 -05:00
25438526ed
HDFS-11033. Add documents for native raw erasure coder in XOR codes. Contributed by SammiChen.
2016-10-21 16:20:07 -07:00
df857f0d10
HDFS-10975. fsck -list-corruptfileblocks does not report corrupt EC files. Contributed by Takanobu Asanuma.
2016-10-21 15:43:12 -07:00
61e30cf83c
HDFS-8410. Add computation time metrics to datanode for ECWorker. Contributed by SammiChen.
2016-10-21 13:12:41 -07:00
ae8bccd509
HADOOP-13702. Add instrumented ReadWriteLock. Contributed by Jingcheng Du
2016-10-21 12:59:54 -07:00
f63cd78f60
HDFS-10730. Fix some failed tests due to BindException. Contributed by Yiqun Lin
2016-10-21 18:16:39 +05:30
d7d87deece
HDFS-10998. Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA. Contributed by Xiaobing Zhou
2016-10-20 19:51:48 -07:00
5e83a21cb6
HDFS-10976. Report erasure coding policy of EC files in Fsck. Contributed by Wei-Chiu Chuang.
2016-10-20 13:06:43 -07:00
6d2da38d16
HDFS-11018. Incorrect check and message in FsDatasetImpl#invalidate. Contributed by Yiqun Lin.
2016-10-20 10:56:39 -07:00
f872c6bc03
HADOOP-13236. truncate will fail when we use viewfilesystem. Contributed by Brahma Reddy Battula
2016-10-20 21:47:17 +05:30
6fb6b651e8
HDFS-10699. Log object instance get incorrectly in TestDFSAdmin. Contributed by Yiqun Lin
2016-10-20 21:28:16 +05:30
5c0bffddc0
HDFS-10627. Volume Scanner marks a block as "suspect" even if the exception is network-related. Contributed by Rushabh S Shah.
2016-10-20 10:48:35 -05:00
4c73be135c
HDFS-9480. Expose nonDfsUsed via StorageTypeStats. Contributed by Brahma Reddy Battula
2016-10-20 20:45:45 +05:30
73504b1bdc
HDFS-11025. TestDiskspaceQuotaUpdate fails in trunk due to Bind exception. Contributed by Yiqun Lin
2016-10-20 10:41:18 +05:30
b4564103e4
HDFS-10752. Several log refactoring/improvement suggestion in HDFS. Contributed by Hanisha Koneru.
2016-10-19 17:20:07 -07:00
c5573e6a75
HDFS-11009. Add a tool to reconstruct block meta file from CLI.
2016-10-18 22:42:28 -07:00
efdf810cf9
HADOOP-7352. FileSystem#listStatus should throw IOE upon access error. Contributed by John Zhuge.
2016-10-18 18:18:43 -07:00
c62ae7107f
HDFS-10906. Add unit tests for Trash with HDFS encryption zones. Contributed by Hanisha Koneru.
2016-10-18 14:05:43 -07:00
d26a1bb9d6
HDFS-10920. TestStorageMover#testNoSpaceDisk is failing intermittently. Contributed by Rakesh R
2016-10-18 14:51:08 +06:00
c023c74886
HADOOP-13061. Refactor erasure coders. Contributed by Kai Sasaki
2016-10-18 12:02:53 +06:00
bedfec0c10
HADOOP-13522. Add %A and %a formats for fs -stat command to print permissions. Contributed by Alex Garbarini.
2016-10-18 15:00:44 +09:00
b61fb267b9
HDFS-9390. Block management for maintenance states.
2016-10-17 17:45:41 -07:00
8fd4c37c45
HDFS-10922. Adding additional unit tests for Trash (II). Contributed by Weiwei Yang.
2016-10-17 14:21:36 -07:00
5f4ae85bd8
HDFS-11003. Expose XmitsInProgress through DataNodeMXBean. Contributed By Brahma Reddy Battula
2016-10-15 22:28:33 +05:30
5ad037df25
HDFS-11012. Unnecessary INFO logging on DFSClients for InvalidToken. Contributed by Harsh J.
...
This closes #142
2016-10-15 22:14:24 +09:00
391ce535a7
HDFS-10301. Remove FBR tracking state to fix false zombie storage detection for interleaving block reports. Contributed by Vinitha Gankidi.
2016-10-14 18:13:54 -07:00
30bb1970cc
HDFS-10558. DiskBalancer: Print the full path to plan file. Contributed by Xiaobing Zhou.
2016-10-14 17:07:59 -07:00
76cc84e6d4
HDFS-11008. Change unit test for testing parsing "-source" parameter in Balancer CLI. Contributed by Mingliang Liu
2016-10-14 14:29:02 -07:00
8c520a27cb
HDFS-10960. TestDataNodeHotSwapVolumes#testRemoveVolumeBeingWritten fails at disk error verification after volume remove. (Manoj Govindassamy via lei)
2016-10-14 13:41:59 -07:00
adb96e109f
HDFS-10827. When there are unrecoverable ec block groups, Namenode Web UI doesn't show the block names. Contributed by Takanobu Asanuma.
2016-10-14 13:21:53 -07:00
0007360c33
HDFS-10883. 's behavior is not consistent in DFS after enabling EZ. Contributed by Yuanbo Liu.
2016-10-14 11:41:29 -07:00
701c27a776
HDFS-10735 Distcp using webhdfs on secure HA clusters fails with StandbyException
2016-10-14 10:26:39 -07:00
8a9f6635a3
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-14 11:38:48 -05:00
8c721aa00a
Revert "HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin."
...
This reverts commit fdce515091
2016-10-13 13:23:28 -07:00
332a61fd74
HDFS-10987. Make Decommission less expensive when lot of blocks present. Contributed by Brahma Reddy Battula.
2016-10-13 13:52:49 -05:00
fdce515091
HDFS-10990. TestPendingInvalidateBlock should wait for IBRs. Contributed by Yiqun Lin.
2016-10-13 11:41:37 -07:00
b371c56365
HDFS-10949. DiskBalancer: deprecate TestDiskBalancer#setVolumeCapacity. Contributed by Xiaobing Zhou.
2016-10-13 10:26:07 -07:00
1291254042
HDFS-10986. DFSAdmin should log detailed error message if any. Contributed by MingLiang Liu
2016-10-13 22:05:00 +05:30
9454dc5e80
HDFS-11000. webhdfs PUT does not work if requests are routed to call queue. Contributed by Kihwal Lee.
2016-10-13 08:47:15 -05:00
901eca004d
HDFS-11002. Fix broken attr/getfattr/setfattr links in ExtendedAttributes.md. Contributed by Mingliang Liu.
2016-10-13 14:29:30 +09:00
85cd06f663
HDFS-10789. Route webhdfs through the RPC call queue. Contributed by Daryn Sharp and Rushabh S Shah.
2016-10-12 15:11:42 -05:00
7ba7092bbc
HDFS-10965. Add unit test for HDFS command 'dfsadmin -printTopology'. Contributed by Xiaobing Zhou
2016-10-11 17:23:54 -07:00
3c9a01062e
HDFS-10903. Replace config key literal strings with config key names II: hadoop hdfs. Contributed by Chen Liang
2016-10-11 16:29:30 -07:00
61f0490a73
HDFS-10984. Expose nntop output as metrics. Contributed by Siddharth Wagle.
2016-10-11 15:55:02 -07:00
dacd3ec66b
HDFS-10991. Export hdfsTruncateFile symbol in libhdfs. Contributed by Surendra Singh Lilhore.
2016-10-11 15:07:14 -07:00
809cfd27a3
HDFS-10916. Switch from "raw" to "system" xattr namespace for erasure coding policy. (Andrew Wang via lei)
2016-10-11 10:04:46 -07:00
96b12662ea
HDFS-10637. Modifications to remove the assumption that FsVolumes are backed by java.io.File. (Virajith Jalaparti via lei)
2016-10-10 15:30:03 -07:00
b963818621
HDFS-10988. Refactor TestBalancerBandwidth. Contributed by Brahma Reddy Battula
2016-10-10 13:19:17 -07:00
3441c746b5
HDFS-10972. Add unit test for HDFS command 'dfsadmin -getDatanodeInfo'. Contributed by Xiaobing Zhou
2016-10-10 11:33:37 -07:00
af50da3298
HDFS-10895. Update HDFS Erasure Coding doc to add how to use ISA-L based coder. Contributed by Sammi Chen
2016-10-10 11:55:49 +06:00
4d106213c0
HDFS-10968. BlockManager#isInNewRack should consider decommissioning nodes. Contributed by Jing Zhao.
2016-10-07 22:44:54 -07:00
6a38d118d8
HDFS-10797. Disk usage summary of snapshots causes renamed blocks to get counted twice. Contributed by Sean Mackrory.
2016-10-07 17:37:15 -07:00
e57fa81d95
HDFS-10980. Optimize check for existence of parent directory. Contributed by Daryn Sharp.
2016-10-07 17:20:15 -05:00
3565c9af17
HDFS-10979. Pass IIP for FSDirDeleteOp methods. Contributed by Daryn Sharp.
2016-10-07 14:15:59 -05:00
3059b251d8
HDFS-10933. Refactor TestFsck. Contributed by Takanobu Asanuma.
2016-10-07 10:17:50 -07:00
be3cb10f53
HDFS-10969. Fix typos in hdfs-default.xml Contributed by Yiqun Lin
2016-10-07 22:18:40 +05:30
48b9d5fd2a
HDFS-10955. Pass IIP for FSDirAttr methods. Contributed by Daryn Sharp.
2016-10-06 16:33:46 -05:00
1d330fbaf6
HADOOP-13150. Avoid use of toString() in output of HDFS ACL shell commands. Contributed by Chris Nauroth.
2016-10-06 13:19:16 -07:00
f32e9fc8f7
HDFS-10939. Reduce performance penalty of encryption zones. Contributed by Daryn sharp.
2016-10-06 15:11:14 -05:00
72a2ae6452
HADOOP-13688. Stop bundling HTML source code in javadoc JARs.
2016-10-06 11:19:38 -07:00
311954883f
HDFS-10957. Retire BKJM from trunk (Vinayakumar B)
2016-10-06 19:28:25 +05:30
b90fc70d67
HDFS-10963. Reduce log level when network topology cannot find enough datanodes. Contributed by Xiao chen
2016-10-06 17:47:31 +05:30
202325485d
HDFS-10893. Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test. Contributed by Mingliang Liu
2016-10-05 22:33:29 -07:00
c5ca216915
HDFS-10683. Make class Token$PrivateToken private. Contributed by John Zhuge.
2016-10-05 17:36:50 -07:00
e68c7b96c7
HDFS-10962. TestRequestHedgingProxyProvider is flaky.
2016-10-05 15:08:56 -07:00
8867762256
HDFS-10826. Correctly report missing EC blocks in FSCK. Contributed by Takanobu Asanuma.
2016-10-05 10:52:50 -07:00
44f48ee96e
HDFS-10956. Remove rename/delete performance penalty when not using snapshots. Contributed by Daryn Sharp.
2016-10-04 15:05:09 -05:00
382307cbdd
HDFS-10878. TestDFSClientRetries#testIdempotentAllocateBlockAndClose throws ConcurrentModificationException. Contributed by Rushabh S Shah.
2016-10-04 11:09:03 -05:00
ef7f06f7d1
HDFS-10947. Correct the API name for truncate in webhdfs document. Contributed by Surendra Singh Lilhore
2016-10-04 18:46:42 +05:30
5ea0210c7f
HDFS-10944. Correct the javadoc of dfsadmin#disallowSnapshot. Contributed by Jagadesh Kiran N
2016-10-04 18:33:22 +05:30
f61e3d13e9
HDFS-10934. TestDFSShell#testStat fails intermittently. Contributed by Eric Badger
2016-10-03 22:34:41 -04:00
8078a5efd0
HDFS-10810. setreplication removing block from underconstrcution temporarily. Contributed by Brahma Reddy Battula
2016-10-03 21:50:16 -04:00
853d65a157
HDFS-10918. Add a tool to get FileEncryptionInfo from CLI. Contributed by Xiao Chen.
2016-10-03 16:01:54 -07:00
607705c488
HDFS-10690. Optimize insertion/removal of replica in ShortCircuitCache. Contributed by Fenghua Hu.
2016-10-03 10:53:21 -07:00
744208431f
HDFS-10940. Reduce performance penalty of block caching when not used. Contributed by Daryn Sharp.
2016-10-03 11:27:23 -05:00
90020624b0
HDFS-10619. Cache path in InodesInPath. Contributed by Daryn Sharp.
2016-10-03 09:13:04 -05:00
fe9ebe20ab
Revert "HDFS-10923. Make InstrumentedLock require ReentrantLock."
...
This reverts commit c7ce6fdc20
2016-09-30 23:11:51 -07:00
c7ce6fdc20
HDFS-10923. Make InstrumentedLock require ReentrantLock.
2016-09-30 23:00:06 -07:00
434c5ea75d
HDFS-10896. Move lock logging logic from FSNamesystem into FSNamesystemLock. Contributed by Erik Krogen.
2016-09-30 13:16:08 -07:00
7fad1221d6
HDFS-10907. Fix Erasure Coding documentation. Contributed by Manoj Govindassamy.
2016-09-30 12:51:27 -07:00
ee33a02234
HDFS-10910. HDFS Erasure Coding doc should state its currently supported erasure coding policies. Contributed by Yiqun Lin.
2016-09-30 12:48:11 -07:00
2ab1ef15c5
HDFS-10908. Improve StripedBlockReader#createBlockReader error logging. Contributed by Manoj Govindassamy.
2016-09-30 12:35:47 -07:00
a0730aa5ce
HDFS-10851. FSDirStatAndListingOp: stop passing path as string. Contributed by Daryn Sharp.
2016-09-30 13:03:24 -05:00
0670149c88
HDFS-10850. getEZForPath should NOT throw FNF. Contributed by Andrew Wang.
2016-09-30 08:44:18 -05:00
84c6264079
HDFS-10892. Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat'. Contributed by Mingliang Liu
2016-09-28 16:03:51 -07:00
92e5e91598
HDFS-10914. Move remnants of oah.hdfs.client to hadoop-hdfs-client.
2016-09-28 16:01:03 -07:00
5f34402ada
HDFS-10779. Rename does not need to re-solve destination. Contributed by Daryn Sharp.
2016-09-28 17:57:23 -05:00
c3b235e565
HDFS-10824. MiniDFSCluster#storageCapacities has no effects on real capacity. Contributed by Xiaobing Zhou.
2016-09-28 11:47:37 -07:00
03f519a757
HDFS-9850. DiskBalancer: Explore removing references to FsVolumeSpi. Contributed by Manoj Govindassamy.
2016-09-27 21:35:06 -07:00
6437ba18c5
HDFS-10915. Fix time measurement bug in TestDatanodeRestart. Contributed by Xiaobing Zhou
2016-09-27 20:40:17 -07:00
9c9736463b
HDFS-10900. DiskBalancer: Complete the documents for the report command. Contributed by Yiqun Lin.
2016-09-27 19:28:41 -07:00
2acfb1e1e4
HDFS-10376. Enhance setOwner testing. (John Zhuge via Yongjun Zhang)
2016-09-27 14:55:28 -07:00
875062b5bc
HADOOP-13544. JDiff reports unncessarily show unannotated APIs and cause confusion while our javadocs only show annotated and public APIs. (vinodkv via wangda)
2016-09-27 11:26:45 -07:00
8ae4729107
HDFS-10828. Fix usage of FsDatasetImpl object lock in ReplicaMap. (Arpit Agarwal)
2016-09-27 10:02:15 -07:00
f55eb981dd
HDFS-10426. TestPendingInvalidateBlock failed in trunk. Contributed by Yiqun Lin.
2016-09-28 01:29:24 +09:00
df1d0f5d56
HDFS-10889. Remove outdated Fault Injection Framework documentaion. Contributed by Brahma Reddy Battula
2016-09-27 19:33:10 +05:30
edf0d0f8b2
HDFS-9444. Add utility to find set of available ephemeral ports to ServerSocketUtil. Contributed by Masatake Iwasaki
2016-09-27 07:04:37 +05:30
059058f961
HDFS-10713. Throttle FsNameSystem lock warnings. Contributed by Hanisha Koneru.
2016-09-26 17:09:47 -07:00
8e06d865c4
Addendum patch for HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications.
2016-09-26 14:41:04 -07:00
3ae652f821
HDFS-10609. Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications. Contributed by Wei-Chiu Chuang.
2016-09-26 13:11:32 -07:00
3e37e243ee
HDFS-10869. Remove the unused method InodeId#checkId(). Contributed by Jagadesh Kiran N
2016-09-25 11:02:46 +05:30
6eb700ee74
HDFS-10866. Fix Eclipse Java compile errors related to generic parameters. Contributed by Konstantin Shvachko.
2016-09-23 14:10:11 -07:00
74b3dd514c
HDFS-10876. Dispatcher#dispatch should log IOException stacktrace. Contributed by Manoj Govindassamy.
2016-09-23 13:26:57 -07:00
a5bb88c8e0
HDFS-10843. Update space quota when a UC block is completed rather than committed. Contributed by Erik Krogen.
2016-09-23 10:37:46 -07:00
bbdf350ff9
HDFS-10886. Replace fs.default.name with fs.defaultFS in viewfs document.
2016-09-23 19:21:34 +05:30
c4480f7f42
HDFS-10894. Remove the redundant charactors for command -saveNamespace in HDFSCommands.md Contributed by Yiqun Lin.
2016-09-23 19:11:02 +05:30
d85d9b2e7b
HDFS-10871. DiskBalancerWorkItem should not import jackson relocated by htrace. Contributed by Manoj Govindassamy.
2016-09-22 19:36:16 -07:00
40acacee08
HDFS-10877. Make RemoteEditLogManifest.committedTxnId optional in Protocol Buffers. Contributed by Sean Mackrory.
2016-09-22 11:43:11 -07:00
964e546ab1
HDFS-9333. Some tests using MiniDFSCluster errored complaining port in use. (iwasakims)
2016-09-21 10:35:25 +09:00
0e918dff59
HDFS-10879. TestEncryptionZonesWithKMS#testReadWrite fails intermittently. Contributed by Xiao Chen.
2016-09-20 16:56:52 -07:00
734d54c1a8
HDFS-10861. Refactor StripeReaders and use ECChunk version decode API. Contributed by Sammi Chen
2016-09-21 21:34:48 +08:00
e52d6e7a46
HDFS-10875. Optimize du -x to cache intermediate result. Contributed by Xiao Chen.
2016-09-19 21:44:42 -07:00
c54f6ef30f
HDFS-10868. Remove stray references to DFS_HDFS_BLOCKS_METADATA_ENABLED.
2016-09-19 11:17:03 -07:00
ea839bd48e
HDFS-10489. Deprecate dfs.encryption.key.provider.uri for HDFS encryption zones. Contributed by Xiao Chen.
2016-09-17 22:25:39 -07:00
8a40953058
HDFS-10823. Implement HttpFSFileSystem#listStatusIterator.
2016-09-16 15:37:36 -07:00
b09a03cd7d
HDFS-10862. Typos in 4 log messages. Contributed by Mehran Hassani.
2016-09-16 16:08:47 +09:00
ec3ea18875
HDFS-9895. Remove unnecessary conf cache from DataNode. Contributed by Xiaobing Zhou.
2016-09-15 15:25:47 -07:00
2a8f55a0cf
HDFS-10805. Reduce runtime for append test. Contributed by Gergely Novak.
2016-09-14 09:31:17 -07:00
ea0c2b8b05
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-15 10:46:00 +08:00
86c9862bec
HDFS-10636. Modify ReplicaInfo to remove the assumption that replica metadata and data are stored in java.io.File. (Virajith Jalaparti via lei)
2016-09-13 12:54:14 -07:00
1c0d18f322
HDFS-10562. DiskBalancer: update documentation on how to report issues and debug. Contributed by Anu Engineer.
2016-09-13 12:42:10 -07:00
db6d243cf8
HDFS-10837. Standardize serializiation of WebHDFS DirectoryListing.
2016-09-13 11:02:36 -07:00
e3f7f58a5f
HDFS-10599. DiskBalancer: Execute CLI via Shell. Contributed by Manoj Govindassamy.
2016-09-13 09:38:12 -07:00
f0876b8b60
HDFS-10856. Update the comment of BPServiceActor$Scheduler#scheduleNextBlockReport. Contributed by Yiqun Lin.
2016-09-13 14:50:03 +09:00
d2466ec3e0
HDFS-10657. testAclCLI.xml setfacl test should expect mask r-x. (John Zhuge via Yongjun Zhang)
2016-09-12 21:25:06 -07:00
72dfb048a9
HDFS-10858. FBR processing may generate incorrect reportedBlock-blockGroup mapping. Contributed by Jing Zhao.
2016-09-12 16:40:11 -07:00
8a93f45a80
HDFS-10821. DiskBalancer: Report command support with multiple nodes. Contributed by Yiqun Lin.
2016-09-12 15:45:26 -07:00
58ed4fa544
HADOOP-13341. Deprecate HADOOP_SERVERNAME_OPTS; replace with (command)_(subcommand)_OPTS
...
This commit includes the following changes:
HADOOP-13356. Add a function to handle command_subcommand_OPTS
HADOOP-13355. Handle HADOOP_CLIENT_OPTS in a function
HADOOP-13554. Add an equivalent of hadoop_subcmd_opts for secure opts
HADOOP-13562. Change hadoop_subcommand_opts to use only uppercase
HADOOP-13358. Modify HDFS to use hadoop_subcommand_opts
HADOOP-13357. Modify common to use hadoop_subcommand_opts
HADOOP-13359. Modify YARN to use hadoop_subcommand_opts
HADOOP-13361. Modify hadoop_verify_user to be consistent with hadoop_subcommand_opts (ie more granularity)
HADOOP-13564. modify mapred to use hadoop_subcommand_opts
HADOOP-13563. hadoop_subcommand_opts should print name not actual content during debug
HADOOP-13360. Documentation for HADOOP_subcommand_OPTS
This closes apache/hadoop#126
2016-09-12 11:10:00 -07:00
cc01ed7026
HDFS-10855. Fix typos in HDFS documents. Contributed by Yiqun Lin.
2016-09-10 23:24:49 -07:00
a99bf26a08
HDFS-10830. FsDatasetImpl#removeVolumes crashes with IllegalMonitorStateException when vol being removed is in use. (Arpit Agarwal and Manoj Govindassamy)
2016-09-10 18:22:25 -07:00
bee9f57f5c
HDFS-10808. DiskBalancer does not execute multi-steps plan-redux. Contributed by Anu Engineer.
2016-09-09 15:00:39 -07:00
cba973f036
HDFS-10832. Propagate ACL bit and isEncrypted bit in HttpFS FileStatus permissions.
2016-09-09 12:12:29 -07:00
a0b0383677
Addendum patch for HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-09 10:54:14 -07:00
baab48922a
HDFS-9849. DiskBalancer: reduce lock path in shutdown code. Contributed by Yuanbo Liu.
2016-09-08 20:00:42 -07:00
35c5943b8b
HDFS-10553. DiskBalancer: Rename Tools/DiskBalancer class to Tools/DiskBalancerCLI. Contributed by Manoj Govindassamy.
2016-09-08 19:26:56 -07:00
b07c266dca
HDFS-10831. Add log when URLConnectionFactory.openConnection failed. Contributed by yunjiong zhao.
2016-09-08 18:30:18 -07:00
011f3b24d4
HDFS-10742. Measure lock time in FsDatasetImpl. Contributed by Chen Liang
2016-09-08 17:53:40 -07:00
b6d839a60c
HDFS-10845. Change defaults in hdfs-site.xml to match timeunit type. Contributed by Yiqun Lin
2016-09-08 13:29:30 -07:00
401db4fc65
HDFS-8901. Use ByteBuffer in striping positional read. Contributed by Sammi Chen and Kai Zheng.
2016-09-08 11:54:33 -07:00
20a20c2f6e
HDFS-10847. Complete the document for FileDistribution processor in OfflineImageViewer. Contributed by Yiqun Lin.
2016-09-09 01:34:34 +09:00
2d1bf53c7e
HDFS-10844. test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failing.
2016-09-09 00:49:22 +09:00
63f594892e
HDFS-10778. Add -format option to make the output of FileDistribution processor human-readable in OfflineImageViewer.
2016-09-08 15:13:43 +09:00
d355573f56
Revert "HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng"
...
This reverts commit 62a9667136
2016-09-08 05:50:17 +08:00
5f23abfa30
HDFS-9038. DFS reserved space is erroneously counted towards non-DFS used. (Brahma Reddy Battula)
2016-09-06 13:37:21 -07:00
f6c0b7543f
HDFS-10841. Remove duplicate or unused variable in appendFile(). Contributed by Kihwal Lee.
2016-09-06 11:48:35 -07:00
7c213c341d
HDFS-10835. Fix typos in httpfs.sh. Contributed by John Zhuge.
2016-09-06 11:44:26 -07:00
f0d5382ff3
HDFS-6962. ACL inheritance conflicts with umaskmode. Contributed by Chris Nauroth.
2016-09-06 11:02:39 -07:00
d37dc5d1b8
HDFS-9847. HDFS configuration should accept time units. Contributed by Yiqun Lin
2016-09-06 10:38:04 -07:00
62a9667136
HADOOP-13218. Migrate other Hadoop side tests to prepare for removing WritableRPCEngine. Contributed by Wei Zhou and Kai Zheng
2016-09-07 17:05:33 +08:00
07650bc37a
HDFS-9781. FsDatasetImpl#getBlockReports can occasionally throw NullPointerException. Contributed by Manoj Govindassamy.
2016-09-02 15:33:11 -07:00
cbd909ce2a
HDFS-10833. Fix JSON errors in WebHDFS.md examples.
2016-09-02 14:50:34 -07:00
5a8c5064d1
HDFS-10822. Log DataNodes in the write pipeline. John Zhuge via Lei Xu
2016-09-02 11:01:13 -07:00
0690f0969e
HDFS-10820. Reuse closeResponder to reset the response variable in DataStreamer#run. Contributed by Yiqun Lin.
2016-09-01 15:35:25 -07:00
6c600360ca
Addendum fix for HDFS-10817 to fix failure of the added testFSReadLockLongHoldingReport in branch-2.
2016-08-31 23:43:59 -07:00
6f4b0d33ca
HDFS-10817. Add Logging for Long-held NN Read Locks. Contributed by Erik Krogen.
2016-08-31 15:40:09 -07:00
85bab5fb57
HDFS-10784. Implement WebHdfsFileSystem#listStatusIterator.
2016-08-31 14:29:37 -07:00
01721dd88e
HDFS-10729. Improve log message for edit loading failures caused by FS limit checks. Contributed by Wei-Chiu Chuang.
2016-08-31 14:02:37 -05:00
20ae1fa259
HDFS-10813. DiskBalancer: Add the getNodeList method in Command. Contributed by Yiqun Lin.
2016-08-30 18:42:55 -07:00
9dcbdbdb5a
HDFS-9392. Admins support for maintenance state. Contributed by Ming Ma.
2016-08-30 14:00:13 -07:00
c4ee6915a1
HDFS-10760. DataXceiver#run() should not log InvalidToken exception as an error. Contributed by Pan Yuxuan.
2016-08-30 10:43:20 -07:00
4bd45f54ee
HDFS-10814. Add assertion for getNumEncryptionZones when no EZ is created. Contributed by Vinitha Reddy Gankidi.
2016-08-29 23:37:26 -07:00
cd5e10ccca
HDFS-4210. Throw helpful exception when DNS entry for JournalNode cannot be resolved. Contributed by Charles Lamb and John Zhuge.
2016-08-29 17:48:08 -07:00
5d1609ddf2
HDFS-10625. VolumeScanner to report why a block is found bad. Contributed by Rushabh S Shah and Yiqun Lin.
2016-08-29 13:59:54 -07:00
6742fb6e68
HDFS-10807. Doc about upgrading to a version of HDFS with snapshots may be confusing. (Contributed by Mingliang Liu)
2016-08-29 10:15:34 -07:00
c25817159a
HDFS-10652. Add a unit test for HDFS-4660. Contributed by Vinayakumar B., Wei-Chiu Chuang, Yongjun Zhang.
2016-08-27 22:51:31 -07:00
e806db7190
HDFS-10584. Allow long-running Mover tool to login with keytab. Contributed by Rakesh R.
2016-08-26 16:43:45 -07:00
a445b82baa
HDFS-10793. Fix HdfsAuditLogger binary incompatibility introduced by HDFS-9184. Contributed by Manoj Govindassamy.
2016-08-26 15:39:18 -07:00
407b519fb1
HDFS-10798. Make the threshold of reporting FSNamesystem lock contention configurable. Contributed by Erik Krogen.
2016-08-26 14:19:55 -07:00
8b7adf4ddf
HDFS-10768. Optimize mkdir ops. Contributed by Daryn Sharp.
2016-08-26 15:39:56 -05:00
13fb1b50e6
HDFS-8915. TestFSNamesystem.testFSLockGetWaiterCount fails intermittently. Contributed by Masatake Iwasaki.
2016-08-26 09:53:10 -05:00
f4a21d3aba
HDFS-10795. Fix an error in ReaderStrategy#ByteBufferStrategy. Contributed by Sammi Chen
2016-08-27 10:54:25 +08:00
4da5000dd3
HDFS-10748. TestFileTruncate#testTruncateWithDataNodesRestart runs sometimes timeout. Contributed by Yiqun Lin.
2016-08-25 09:00:44 -07:00
a1f3293762
HDFS-10772. Reduce byte/string conversions for get listing. Contributed by Daryn Sharp.
2016-08-24 15:21:08 -05:00
793447f799
HDFS-8905. Refactor DFSInputStream#ReaderStrategy. Contributed by Kai Zheng and Sammi Chen
2016-08-24 21:57:23 +08:00
ec252ce0fc
HDFS-10762. Pass IIP for file status related methods
2016-08-24 08:46:47 -05:00
126d165efd
HDFS-9745. TestSecureNNWithQJM#testSecureMode sometimes fails with timeouts. Contributed by Xiao Chen
2016-08-23 14:46:58 +00:00
f0efea490e
HDFS-8986. Add option to -du to calculate directory space usage excluding snapshots. Contributed by Xiao Chen.
2016-08-23 04:14:24 -07:00
e90f3359de
HDFS-10783. The option '-maxSize' and '-step' fail in OfflineImageViewer. Contributed by Yiqun Lin.
2016-08-23 19:57:23 +09:00
c49333becf
HDFS-8312. Added permission check for moving file to Trash. (Weiwei Yang via Eric Yang)
2016-08-22 18:29:56 -07:00
3ca4d6ddfd
Revert "HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp."
...
This reverts commit 22fc46d765
2016-08-22 16:57:45 -05:00
22fc46d765
HDFS-10762. Pass IIP for file status related methods. Contributed by Daryn Sharp.
2016-08-22 15:37:02 -05:00
dc7a1c54f4
HDFS-10692. Update JDiff report's base version for HDFS from 2.6.0 to 2.7.2. Contributed by Wangda Tan.
2016-08-22 11:35:18 -07:00
0faee62a0c
HDFS-10764. Fix INodeFile#getBlocks to not return null. Contributed by Arpit Agarwal.
2016-08-19 22:13:36 -07:00
03a9343d57
HADOOP-12765. HttpServer2 should switch to using the non-blocking SslSelectChannelConnector to prevent performance degradation when handling SSL connections. Contributed by Min Shen.
2016-08-19 09:22:49 -07:00
Arpit Agarwal
Mingliang Liu
Kihwal Lee
Zhe Zhang
Brahma Reddy Battula
Xiaoyu Yao
Akira Ajisaka
Vinayakumar B
Xiao Chen
Wei-Chiu Chuang
Andrew Wang
Anu Engineer
Lei Xu
Kai Zheng
Robert Kanter
Erik Krogen
Masatake Iwasaki
Chris Douglas
Ming Ma
Vinitha Reddy Gankidi
Jing Zhao
Benoy Antony
Chris Nauroth
Yongjun Zhang
Wangda Tan
Konstantin V Shvachko
Allen Wittenauer
Daryn Sharp
Jason Lowe
Eric Yang
Vinod Kumar Vavilapalli (I am also known as @tshooter.)