Commit Graph

6858 Commits

Author SHA1 Message Date
Arpit Agarwal bbc6d254c8 HDFS-12224. Add tests to TestJournalNodeSync for sync after JN downtime. Contributed by Hanisha Koneru. 2017-08-04 12:51:33 -07:00
Xiaoyu Yao 5fd34ba195 HDFS-12163. Ozone: Reduce MiniOzoneCluster handler thread count. Contributed by Weiwei Yang. 2017-08-04 11:33:10 -07:00
Lei Xu fe3341786a HDFS-12251. Add document for StreamCapabilities. (Lei (Eddy) Xu) 2017-08-04 11:21:58 -07:00
Xiaoyu Yao 15101eef75 HDFS-12246. Ozone: potential thread leaks. Contributed by Weiwei Yang. 2017-08-04 11:11:11 -07:00
Anu Engineer af6dfdf7ee HDFS-11935. Ozone: TestStorageContainerManager#testRpcPermission fails when ipv6 address used. Contributed by Yiqun Lin. 2017-08-04 10:32:32 -07:00
Anu Engineer 72e3016224 HDFS-12178. Ozone: OzoneClient: Handling SCM container creationFlag at client side. Contributed by Nandakumar. 2017-08-04 10:17:45 -07:00
Andrew Wang f4c6b00a9f HDFS-12131. Add some of the FSNamesystem JMX values as metrics. Contributed by Erik Krogen. 2017-08-03 15:44:51 -07:00
Xiaoyu Yao 507447c816 HDFS-12247. Ozone: KeySpaceManager should unregister KSMMetrics upon stop. Contributed by Yiqun Lin. 2017-08-03 12:45:46 -07:00
Manoj Govindassamy 79df1e750e HDFS-9388. Decommission related code to support Maintenance State for datanodes. 2017-08-02 14:22:46 -07:00
Chris Douglas 12e44e7bda HDFS-6984. Serialize FileStatus via protobuf. 2017-08-02 12:12:48 -07:00
Sean Mackrory 1a1bf6b7d0 HADOOP-13595. Rework hadoop_usage to be broken up by clients/daemons/etc. Contributed by Allen Wittenauer. 2017-08-02 12:25:05 -06:00
Haohui Mai 6814324c33 HDFS-12107. FsDatasetImpl#removeVolumes floods the logs when removing the volume. Contributed by Kelvin Chu. 2017-08-01 18:34:07 -07:00
Manoj Govindassamy 52d7bafcf4 HDFS-12217. HDFS snapshots doesn't capture all open files when one of the open files is deleted. 2017-08-01 16:28:20 -07:00
Sean Mackrory 7774759830 HDFS-12151. Hadoop 2 clients cannot writeBlock to Hadoop 3 DataNodes. 2017-08-01 14:34:25 -06:00
Anu Engineer 92945d01b2 HDFS-11580. Ozone: Support asynchronus client API for SCM and containers. Contributed by Anu Engineer. 2017-08-01 12:32:11 -07:00
Akira Ajisaka b38a1eea8e
HADOOP-14245. Use Mockito.when instead of Mockito.stub. Contributed by Andras Bokor. 2017-08-01 15:15:43 +09:00
Weiwei Yang 1fd2790344 HDFS-11984. Ozone: Ensures listKey lists all required key fields. Contributed by Yiqun Lin. 2017-08-01 13:47:44 +08:00
Lei Xu 9586b0e24f HADOOP-14397. Pull up the builder pattern to FileSystem and add AbstractContractCreateTest for it. (Lei (Eddy) Xu) 2017-07-31 20:12:40 -07:00
Weiwei Yang ea568123fa HDFS-12154. Incorrect javadoc description in StorageLocationChecker#check. Contributed by Nandakumar. 2017-07-31 17:02:44 -07:00
Anu Engineer c49297b5b2 HDFS-12034. Ozone: Web interface for KSM. Contributed by Elek, Marton. 2017-07-31 16:05:21 -07:00
Chen Liang 42ab44d34e HDFS-11920. Ozone : add key partition. Contributed by Chen Liang. 2017-07-31 16:01:39 -07:00
Arpit Agarwal 3e23415a92 HDFS-12082. BlockInvalidateLimit value is incorrectly set after namenode heartbeat interval reconfigured. Contributed by Weiwei Yang. 2017-07-31 11:33:55 -07:00
Anu Engineer 1f5353d7b6 HDFS-12195. Ozone: DeleteKey-1: KSM replies delete key request asynchronously. Contributed by Yuanbo Liu. 2017-07-31 10:23:11 -07:00
Weiwei Yang 6fb800fbe0 HDFS-12179. Ozone: Corona: Adding corona as part of hdfs command. Contributed by Nandakumar. 2017-07-29 08:23:05 +08:00
Anu Engineer e269865a2b HDFS-12183. Ozone: KSM : Use proper defaults for block client address. Contributed by Lokesh Jain. 2017-07-28 15:40:59 -07:00
Andrew Wang 480c8db40c HDFS-12206. Rename the split EC / replicated block metrics. 2017-07-28 11:24:44 -07:00
Lei Xu 77791e4c36 HDFS-12044. Mismatch between BlockManager.maxReplicationStreams and ErasureCodingWorker.stripedReconstructionPool pool size causes slow and bursty recovery. (Contributed by Lei (Eddy) Xu) 2017-07-28 10:50:49 -07:00
Wei-Chiu Chuang 369f731264 HDFS-12062. removeErasureCodingPolicy needs super user permission. Contributed by Wei-Chiu Chuang. 2017-07-28 00:51:03 -07:00
Yongjun Zhang c6330f22a5 HDFS-12190. Enable 'hdfs dfs -stat' to display access time. Contributed by Yongjun Zhang. 2017-07-27 16:48:24 -07:00
Anu Engineer 52640fb884 Merge branch 'trunk' into HDFS-7240 2017-07-27 16:44:36 -07:00
Jitendra Pandey e3c7300225 HDFS-2319. Add test cases for FSshell -stat. Contributed by XieXianshan and Bharat Viswanadham. 2017-07-27 13:23:15 -07:00
Brahma Reddy Battula c4a85c694f HDFS-11896. Non-dfsUsed will be doubled on dead node re-registration. Contributed by Brahma Reddy Battula. 2017-07-27 12:02:57 -07:00
Chen Liang 5e47076032 HDFS-12059. Ozone: OzoneClient: OzoneClientImpl Add setBucketProperty and delete calls. Contributed by Nandakumar. 2017-07-27 10:39:33 -07:00
Anu Engineer 5b3cac7657 HDFS-12180. Ozone: Corona: Add stats and progress bar to corona. Contributed by Nandakumar. 2017-07-26 22:48:06 -07:00
Weiwei Yang c651868c4a HDFS-12145. Ozone: OzoneFileSystem: Ozone & KSM should support / delimited key names. Contributed by Mukul Kumar Singh.
(cherry picked from commit fe0a957e92d506330ba005abba18f665eb5ac1cc)
2017-07-26 22:36:23 -07:00
Yiqun Lin f6ed186330 HDFS-12170. Ozone: OzoneFileSystem: KSM should maintain key creation time and modification time. Contributed by Mukul Kumar Singh. 2017-07-27 11:43:01 +08:00
Weiwei Yang 04b39066db HDFS-12192. Ozone: Fix the remaining failure tests for Windows caused by incorrect path generated. Contributed by Yiqun Lin. 2017-07-26 17:51:05 +08:00
Weiwei Yang 5d0f545143 HDFS-12155. Ozone : add RocksDB support to DEBUG CLI. Contributed by Chen Liang. 2017-07-26 12:59:25 +08:00
Anu Engineer c2d08e6013 HDFS-12115. Ozone: SCM: Add queryNode RPC Call. Contributed by Anu Engineer. 2017-07-25 13:28:33 -07:00
Kihwal Lee a68b5b31cf HDFS-12171. Reduce IIP object allocations for inode lookup. Contributed by Daryn Sharp. 2017-07-25 11:03:49 -05:00
Brahma Reddy Battula 6d983cca52 HADOOP-14455. ViewFileSystem#rename should support be supported within same nameservice with different mountpoints. Contributed by Brahma Reddy Battula. 2017-07-25 23:51:53 +08:00
Kihwal Lee 1a79dcfc45 HDFS-12143. Improve performance of getting and removing inode features. Contributed by Daryn Sharp. 2017-07-25 10:28:57 -05:00
Ravi Prakash c98201b5d8 HDFS-12193. Fix style issues in HttpFS tests. Contributed by Zoran Dimitrijevic 2017-07-24 19:06:15 -07:00
Weiwei Yang f35def4cc8 HDFS-12187. Ozone : add support to DEBUG CLI for ksm.db. Contributed by Chen Liang. 2017-07-25 09:43:19 +08:00
Akira Ajisaka 770cc46228
HDFS-12176. dfsadmin shows DFS Used%: NaN% if the cluster has zero block. Contributed by Weiwei Yang. 2017-07-24 16:24:23 +09:00
Weiwei Yang c539095200 HDFS-12167. Ozone: Intermittent failure TestContainerPersistence#testListKey. Contributed by Surendra Singh Lilhore. 2017-07-22 22:02:35 +08:00
Anu Engineer 63e211ff75 HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao. 2017-07-21 21:06:37 -07:00
Anu Engineer 91b089c8e3 HDFS-12175. Ozone: Fix Leaking in TestStorageContainerManager#testRpcPermission. Contributed by Xiaoyu Yao. 2017-07-21 21:06:12 -07:00
Chen Liang c081e831d7 Revert "HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao"
This reverts commit f801454ad3.
2017-07-21 14:21:52 -07:00
Xiaoyu Yao 4ddae60c89 HDFS-12181. Ozone: Fix TestContainerReplicationManager by setting proper log level for LogCapturer. Contributed by Mukul Kumar Singh. 2017-07-21 14:20:20 -07:00
Chen Liang f801454ad3 HDFS-12185. Ozone: Fix Leaking in TestXceiverClientManager. Contributed by Xiaoyu Yao 2017-07-21 14:16:01 -07:00
Kihwal Lee 8e3a992ecc HDFS-11742. Improve balancer usability after HDFS-8818. Contributed by Kihwal Lee 2017-07-21 09:14:19 -05:00
Weiwei Yang b92587c35d HDFS-12127. Ozone: Ozone shell: Add more testing for key shell commands. Contributed by Yiqun Lin. 2017-07-21 20:43:57 +08:00
Weiwei Yang e5f2c5f893 HDFS-11936. Ozone: TestNodeManager times out before it is able to find all nodes. Contributed by Yuanbo Liu. 2017-07-21 15:32:27 +08:00
John Zhuge 3b48f81411 HDFS-12166. Do not deprecate HTTPFS_TEMP. Contributed by John Zhuge.
Change-Id: Icdd032de9feea3d0dde71bf8145b70497993d97f
2017-07-20 20:45:09 -07:00
Anu Engineer 0edc4c3dd8 HDFS-12149. Ozone: RocksDB implementation of ozone metadata store. Contributed by Weiwei Yang. 2017-07-20 18:22:03 -07:00
Anu Engineer c2f2e0fa2f HDFS-12071. Ozone: Corona: Implementation of Corona. Contributed by Nandakumar. 2017-07-20 17:59:53 -07:00
Konstantin V Shvachko 8c2c812832 HDFS-11472. Fix inconsistent replica size after a data pipeline failure. Contributed by Erik Krogen and Wei-Chiu Chuang. 2017-07-20 14:47:25 -07:00
Xiaoyu Yao b3a7f3b2df Merge branch 'trunk' into HDFS-7240 2017-07-19 14:52:35 -07:00
Xiaoyu Yao 84e11c7c01 Merge branch 'HDFS-7240' of https://git-wip-us.apache.org/repos/asf/hadoop into HDFS-7240 2017-07-19 14:47:05 -07:00
Yongjun Zhang 3556e36be3 HDFS-12139. HTTPFS liststatus returns incorrect pathSuffix for path of file. Contributed by Yongjun Zhang. 2017-07-19 10:56:50 -07:00
Anu Engineer 413b23eb04 HDFS-12158. Secondary Namenode's web interface lack configs for X-FRAME-OPTIONS protection. Contributed by Mukul Kumar Singh. 2017-07-19 10:29:06 -07:00
Brahma Reddy Battula 04ff412dab HDFS-12133. Correct ContentSummaryComputationContext Logger class name.. Contributed by Surendra Singh Lilhore. 2017-07-19 23:43:10 +08:00
Brahma Reddy Battula f8cd55fe33 HDFS-12067. Correct dfsadmin commands usage message to reflects IPC port. Contributed by steven-wugang. 2017-07-19 23:21:43 +08:00
Weiwei Yang a715f60ce1 HDFS-12126. Ozone: Ozone shell: Add more testing for bucket shell commands. Contributed by Yiqun Lin. 2017-07-19 14:05:54 +08:00
Weiwei Yang bb0adcb6e0 HDFS-12154. Incorrect javadoc description in StorageLocationChecker#check. Contributed by Nandakumar. 2017-07-18 17:28:08 +08:00
Weiwei Yang 0981119eb3 HDFS-11996. Ozone : add an UT to test partial read of chunks. Contributed by Chen Liang. 2017-07-18 12:12:20 +08:00
Tsz-Wo Nicholas Sze ed27f2b2cc HDFS-12138. Remove redundant 'public' modifiers from BlockCollection. Contributed by Chen Liang 2017-07-17 13:54:16 -07:00
Anu Engineer e0687dde4a HDFS-12058. Ozone: OzoneClient: OzoneClientImpl add getDetails calls. Contributed by Nandakumar. 2017-07-17 12:59:47 -07:00
Chen Liang 19f665f480 HDFS-12118. Ozone: Ozone shell: Add more testing for volume shell commands. Contributed by Yiqun Lin. 2017-07-17 11:35:15 -07:00
Weiwei Yang 1bec6a11a2 HDFS-12148. Ozone: TestOzoneConfigurationFields is failing because ozone-default.xml has some missing properties. Contributed by Weiwei Yang. 2017-07-16 22:41:45 +08:00
Weiwei Yang 8f122a7505 HDFS-12069. Ozone: Create a general abstraction for metadata store. Contributed by Weiwei Yang. 2017-07-16 09:34:02 +08:00
Brahma Reddy Battula b778887af5 HDFS-12112. TestBlockManager#testBlockManagerMachinesArray sometimes fails with NPE. Contributed by Wei-Chiu Chuang. 2017-07-15 10:38:31 +08:00
Xiao Chen 0a6d5c0cf1 HADOOP-14521. KMS client needs retry logic. Contributed by Rushabh S Shah. 2017-07-14 22:22:53 -07:00
Anu Engineer 90f1d58546 HDFS-11989. Ozone: add TestKeysRatis, TestBucketsRatis and TestVolumeRatis. Contributed by Tsz Wo Nicholas Sze. 2017-07-14 17:33:05 -07:00
Anu Engineer c99d2890e8 HDFS-12123. Ozone: OzoneClient: Abstraction of OzoneClient and default implementation. Contributed by Nandakumar. 2017-07-14 15:28:50 -07:00
Tsz-Wo Nicholas Sze f413ee33df HDFS-12130. Optimizing permission check for getContentSummary. Contributed by Chen Liang 2017-07-14 14:35:51 -07:00
Tsz-Wo Nicholas Sze a1f12bb543 Revert "HDFS-12130. Optimizing permission check for getContentSummary." to fix commit message.
This reverts commit a29fe100b3.
2017-07-14 14:34:01 -07:00
Kihwal Lee e7d187a1b6 HDFS-12140. Remove BPOfferService lock contention to get block pool id. Contributed by Daryn Sharp. 2017-07-14 16:07:17 -05:00
Kihwal Lee 8d86a93915 HDFS-12137. DN dataset lock should be fair. Contributed by Daryn Sharp. 2017-07-14 15:41:43 -05:00
Tsz-Wo Nicholas Sze a29fe100b3 HDFS-12130. Optimizing permission check for getContentSummary. 2017-07-14 13:36:27 -07:00
Anu Engineer 8d37ef30e1 HDFS-11493. Ozone: SCM: Add the ability to handle container reports. Contributed by Anu Engineer. 2017-07-14 11:18:36 -07:00
Yiqun Lin 5341fa13af HDFS-12083. Ozone: KSM: previous key has to be excluded from result in listVolumes, listBuckets and listKeys. Contributed by Nandakumar. 2017-07-14 23:52:34 +08:00
Weiwei Yang 35c55ebd0e HDFS-12129. Ozone: SCM http server is not stopped with SCM#stop(). Contributed by Weiwei Yang. 2017-07-13 16:10:58 +08:00
Weiwei Yang ef9ba8332c HDFS-12105. Ozone: listVolumes doesn't work from ozone commandline. Contributed by Yiqun Lin. 2017-07-12 17:11:46 -07:00
Weiwei Yang 6798e6dd71 HDFS-12037. Ozone: Improvement rest API output format for better looking. Contributed by Weiwei Yang. 2017-07-12 17:11:46 -07:00
Xiaoyu Yao e15e2713e1 HDFS-11502. Datanode UI should display hostname based on JMX bean instead of window.location.hostname. Contributed by Jeffrey E Rodriguez and Yuanbo Liu. 2017-07-12 16:26:19 -07:00
Tsz-Wo Nicholas Sze 931a49800e HDFS-6874. Add GETFILEBLOCKLOCATIONS operation to HttpFS. Contributed by Weiwei Yang 2017-07-12 14:15:04 -07:00
John Zhuge ac0a04a6e1 HDFS-12114. Consistent HttpFS property names. Contributed by John Zhuge. 2017-07-11 21:06:41 -07:00
Akira Ajisaka 9144fd9e9b
HADOOP-14629. Improve exception checking in FileContext related JUnit tests. Contributed by Andras Bokor. 2017-07-12 11:35:50 +09:00
John Zhuge 12c8fdceaf HDFS-12052. Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS. Contributed by Zoran Dimitrijevic. 2017-07-11 11:19:08 -07:00
Yiqun Lin 3a7f02b815 HDFS-12085. Reconfigure namenode heartbeat interval fails if the interval was set with time unit. Contributed by Weiwei Yang. 2017-07-11 20:40:11 +08:00
Weiwei Yang 87154fcd61 HDFS-12105. Ozone: listVolumes doesn't work from ozone commandline. Contributed by Yiqun Lin. 2017-07-10 10:24:22 +08:00
Weiwei Yang 4e3fbc8697 HDFS-12037. Ozone: Improvement rest API output format for better looking. Contributed by Weiwei Yang. 2017-07-08 10:06:58 +08:00
Xiaoyu Yao b36d08633a Merge branch 'HDFS-7240' of https://git-wip-us.apache.org/repos/asf/hadoop into HDFS-7240 2017-07-07 13:17:11 -07:00
Xiaoyu Yao 99474212cc Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into HDFS-7240 2017-07-07 13:16:18 -07:00
Andrew Wang f10864a820 Add release notes, changes, jdiff for 3.0.0-alpha4 2017-07-07 11:01:59 -07:00
Akira Ajisaka 8fc5dcc2a1
Revert "HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He."
This reverts commit 82cb2a6497.
2017-07-08 02:53:18 +09:00
Weiwei Yang 5fd38a6dc1 HDFS-11679. Ozone: SCM CLI: Implement list container command. Contributed by Yuanbo Liu. 2017-07-07 15:54:21 +08:00
Akira Ajisaka 82cb2a6497
HADOOP-14587. Use GenericTestUtils.setLogLevel when available in hadoop-common. Contributed by Wenxin He. 2017-07-07 15:00:47 +09:00