Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,923 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

257 Commits

Author SHA1 Message Date
Shashikant Banerjee de1dae64f2 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee. 2019-03-01 23:30:28 +05:30
Bharat Viswanadham 77b23c816e
HDDS-1182. Pipeline Rule where at least one datanode is reported in the pipeline. 2019-02-28 21:41:36 -08:00
Bharat Viswanadham eae8819fd2
HDDS-1187. Healthy pipeline Chill Mode rule to consider only pipelines with replication factor three. 2019-02-28 17:03:39 -08:00
Bharat Viswanadham 6c8c422beb
HDDS-1178. Healthy pipeline Chill Mode Rule. 
Closes #518
 2019-02-27 14:26:40 +01:00
Xiaoyu Yao 9e0f3d1c52 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar. 2019-02-26 06:02:10 -08:00
Anu Engineer 9de34d2990 HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol. 
Contributed by Elek, Marton.
 2019-02-25 13:50:37 -08:00
Anu Engineer 92b1fdcece HDDS-1070. Adding Node and Pipeline related metrics in SCM. 
Contributed by Nandakumar.
 2019-02-24 15:39:41 -08:00
Anu Engineer 8387bbdfb5 HDDS-1154. Add tracing to the client side of StorageContainerLocationProtocol and OzoneManagerProtocol. 
Contributed by Elek, Marton.
 2019-02-24 13:13:37 -08:00
Anu Engineer 021f8deef0 HDDS-1168.Use random ports in TestBlockManager and TestDeletedBlockLog. 
Contributed by Nandakumar.
 2019-02-24 12:36:03 -08:00
Anu Engineer 490206e4b4 HDDS-1155.Fix failing unit test methods of TestDeadNodeHandler. 
Contributed by Nandakumar.
 2019-02-23 21:13:47 -08:00
Anu Engineer 014e17af78 HDDS-1166. Fix checkstyle line length issues. 
Contributed by Nandakumar.
 2019-02-23 20:31:39 -08:00
Bharat Viswanadham 70579805c9 HDDS-1148. After allocating container, we are not adding to container DB. 2019-02-23 01:50:49 +05:30
Elek, Márton 28d0bf9f67 HDDS-1161. Disable failing test which are tracked by a separated jira. Contributed by Elek, Marton. 2019-02-21 13:57:05 -08:00
Márton Elek 7c802c42dc
HDDS-1129. Fix findbug/checkstyle errors hdds projects. Contributed by Elek, Marton. 2019-02-21 14:04:18 +01:00
Xiaoyu Yao 1374f8f548 HDDS-1060. Add API to get OM certificate from SCM CA. Contributed by Ajay Kumar. 2019-02-20 11:35:49 -08:00
Xiaoyu Yao cf1a66d0da HDDS-1101. SCM CA: Write Certificate information to SCM Metadata. Contributed by Anu Engineer. 2019-02-19 10:42:56 -08:00
Yiqun Lin f2fb6536dc HDDS-1106. Introduce queryMap in PipelineManager. Contributed by Lokesh Jain. 2019-02-18 22:35:23 +08:00
Shashikant Banerjee de934ba2dc HDDS-1076. TestSCMNodeManager crashed the jvm. Contributed by Lokesh Jain. 2019-02-15 21:13:30 +05:30
Shashikant Banerjee 8a426dc848 HDDS-1028. Improve logging in SCMPipelineManager. Contributed by Lokesh Jain. 2019-02-15 14:38:38 +05:30
Anu Engineer 75e15cc0c4 HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects. 
Contributed by Elek, Marton.
 2019-02-14 23:33:25 -08:00
Nanda kumar 4f7d32edf8 HDDS-1081. CLOSING state containers should not be added to pipeline on SCM start. 
Contributed by Lokesh Jain.
 2019-02-12 14:39:08 +05:30
Nanda kumar a536eb5c41 HDDS-360. Use RocksDBStore and TableStore for SCM Metadata. 
Contributed by Anu Engineer.
 2019-02-12 14:26:19 +05:30
Márton Elek e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Yiqun Lin fb8c997a68 HDDS-1048. Remove SCMNodeStat from SCMNodeManager and use storage information from DatanodeInfo#StorageReportProto. Contributed by Nanda kumar. 2019-02-08 23:49:37 +08:00
Nanda kumar 0c1bc4dcee HDDS-981. Block allocation should involve pipeline selection and then container selection. 
Contributed by Lokesh Jain.
 2019-02-08 15:44:41 +05:30
Yiqun Lin 3efa168e1f HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham. 2019-02-05 10:51:52 +08:00
Mukul Kumar Singh b6f90d3957 HDDS-1021. Allow ozone datanode to participate in a 1 node as well as 3 node Ratis Pipeline. Contributed by Shashikant Banerjee. 2019-02-03 00:53:06 +05:30
Yiqun Lin 16195eaee1 HDDS-1025. Handle replication of closed containers in DeadNodeHanlder. Contributed by Bharat Viswanadham. 2019-02-01 11:34:51 +08:00
Márton Elek 14441ccbc6 HDDS-1032. Package builds are failing with missing org.mockito:mockito-core dependency version. Contributed by Doroszlai, Attila. 2019-01-30 13:40:45 +01:00
Yiqun Lin d583cc45c6 HDDS-1024. Handle DeleteContainerCommand in the SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-30 13:56:28 +08:00
Akira Ajisaka 1129288cf5
HADOOP-14178. Move Mockito up to version 2.23.4. Contributed by Akira Ajisaka and Masatake Iwasaki. 2019-01-29 18:29:56 -08:00
Shashikant Banerjee 5d578d0c4a HDDS-1004. SCMContainerManager#updateContainerStateInternal fails for QUASI_CLOSE and FORCE_CLOSE events. Contributed by Lokesh Jain. 2019-01-29 14:11:56 +05:30
Yiqun Lin 2d06112b74 HDDS-1022. Add cmd type in getCommandResponse in SCMDatanodeProtocolServer. Contributed by Bharat Viswanadham. 2019-01-29 11:26:41 +08:00
Nanda kumar 45c4cfe790 HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. 
Contributed by Doroszlai, Attila.
 2019-01-25 12:16:29 +05:30
Xiaoyu Yao e321b91cb5 HDDS-980. Adding getOMCertificate in SCMSecurityProtocol. Contributed by Ajay Kumar. 2019-01-23 19:32:03 -08:00
Ajay Kumar 01a7f9ed35 HDDS-597. Ratis: Support secure gRPC endpoint with mTLS for Ratis. Contributed by Ajay Kumar. 2019-01-17 10:30:44 -08:00
Ajay Kumar 0e16cf1973 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar. 2019-01-16 16:40:05 -07:00
Anu Engineer 30bfc9cbd0 HDDS-955. SCM CA: Add CA to SCM. 
Contributed by Anu Engineer.
 2019-01-16 13:43:31 -07:00
Ajay Kumar a28ad7ad81 HDDS-753. SCM security protocol server is not starting. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar 33c274ea2e HDDS-103. SCM CA: Add new security protocol for SCM to expose security related functions. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar 9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Xiaoyu Yao 570b503e3e Fix merge conflicts 2019-01-15 22:54:50 -07:00
Xiaoyu Yao e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao ff61931f91 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:16:58 -07:00
Márton Elek 32cf0412ce HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton. 2019-01-08 22:27:19 +01:00
Anu Engineer ddc0a40507 HDDS-896. Handle over replicated containers in SCM. 
Contributed by Nandakumar.
 2019-01-04 09:04:10 -08:00
Bharat Viswanadham 26e4be7022 HDDS-921. Add JVM pause monitor to Ozone Daemons (OM, SCM and Datanodes). Contributed by Bharat Viswanadham. 2018-12-23 17:33:25 -08:00
Arpit Agarwal cf571133b8 HDDS-893. pipeline status is ALLOCATED in scmcli listPipelines command. Contributed by Lokesh Jain. 2018-12-19 13:42:06 +05:30
Shashikant Banerjee b51e9e4317 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain. 2018-12-18 18:21:17 +05:30
Xiaoyu Yao 94b368f29f HDDS-99. Adding SCM Audit log. Contributed by Dinesh Chitlangia. 2018-12-17 15:40:22 -08:00
Yiqun Lin 3044b78bd0 HDDS-848. Create SCM metrics related to container state. Contributed by Bharat Viswanadham. 2018-12-03 17:16:34 +08:00
Ajay Kumar b71cc7f33e HDDS-642. Add chill mode exit condition for pipeline availability. Contributed by Yiqun Lin. 2018-11-28 17:47:57 -08:00
Nanda kumar 099e723406 HDDS-868. Handle quasi closed container replicas in SCM. 
Contributed by Nanda kumar.
 2018-11-27 13:01:22 +05:30
Xiaoyu Yao 919a6e4310 HDDS-861. SCMNodeManager unit tests are broken. Contributed by Xiaoyu Yao. 2018-11-21 11:46:53 -08:00
Yiqun Lin d0cc679441 HDDS-817. Create SCM metrics for disk from node report. Contributed by Bharat Viswanadham. 2018-11-20 14:22:30 +08:00
Mukul Kumar Singh b5d7b292c9 HDDS-718. Introduce new SCM Commands to list and close Pipelines. Contributed by Lokesh Jain. 2018-11-19 22:59:30 +05:30
Nanda kumar 5a7ca6ac3e HDDS-837. Persist originNodeId as part of .container file in datanode. 
Contributed by Nanda kumar.
 2018-11-19 14:39:55 +05:30
Mukul Kumar Singh cfb915f3df HDDS-844. Add logic for pipeline teardown after timeout. Contributed by Lokesh Jain. 2018-11-19 12:11:05 +05:30
Nanda kumar c4d9640028 HDDS-801. Quasi close the container when close is not executed via Ratis. 
Contributed by Nanda kumar.
 2018-11-16 23:11:33 +05:30
Anu Engineer a16aa2f60b HDDS-825. Code cleanup based on messages from ErrorProne. 
Contributed by Anu Engineer.
 2018-11-15 17:36:09 -08:00
Giovanni Matteo Fumarola d8ec017a8d HDDS-843. [JDK11] Fix Javadoc errors in hadoop-hdds-server-scm module. Contributed by Dinesh Chitlangia. 2018-11-15 14:54:41 -08:00
Márton Elek 0a8b1efbfb HDDS-827. TestStorageContainerManagerHttpServer should use dynamic port. Contributed by Sandeep Nemuri. 2018-11-15 14:23:24 +01:00
Anu Engineer 9b01f039eb HDDS-832. Docs folder is missing from the Ozone distribution package. Contributed by Elek, Marton. 2018-11-15 11:08:48 +01:00
Nanda kumar 18fe65d756 HDDS-576. Move ContainerWithPipeline creation to RPC endpoint. 
Contributed by Nanda kumar.
 2018-11-12 23:33:11 +05:30
Nanda kumar 9fe50b4991 HDDS-733. Create container if not exist, as part of chunk write. 
Contributed by Lokesh Jain.
 2018-11-10 00:08:17 +05:30
Bharat Viswanadham 66bf62404a HDDS-812. TestEndPoint#testCheckVersionResponse is failing. Contributed by Hanisha Koneru. 2018-11-08 17:52:09 -08:00
Yiqun Lin 9317a61f3c HDDS-802. Container State Manager should get open pipelines for allocating container. Contributed by Lokesh Jain. 2018-11-08 23:41:43 +08:00
Nanda kumar c80f753b0e HDDS-737. Introduce Incremental Container Report. 
Contributed by Nanda kumar.
 2018-11-08 18:33:38 +05:30
Yiqun Lin addec29297 HDDS-809. Refactor SCMChillModeManager. 2018-11-07 13:54:08 +08:00
Yiqun Lin 2e8ac14dcb HDDS-786. Fix the findbugs for SCMClientProtocolServer#getContainerWithPipeline. 2018-11-01 14:10:17 +08:00
Arpit Agarwal 08bb0362e0 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal. 2018-10-31 11:23:15 -07:00
Nanda kumar e4f22b08e0 HDDS-755. ContainerInfo and ContainerReplica protobuf changes. 
Contributed by Nanda kumar.
 2018-10-31 10:29:35 +05:30
Shashikant Banerjee 7757331dbc HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee. 2018-10-30 14:15:27 +05:30
Arpit Agarwal 496f0ffe90 HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal. 2018-10-29 17:14:18 -07:00
Nanda kumar 902345de66 HDDS-728. Datanodes should use different ContainerStateMachine for each pipeline. 
Contributed by Mukul Kumar Singh.
 2018-10-29 19:53:52 +05:30
Nanda kumar dce4ebe814 HDDS-694. Plugin new Pipeline management code in SCM. 
Contributed by Lokesh Jain.
 2018-10-26 17:55:04 +05:30
Arpit Agarwal 244afaba4a HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal. 2018-10-24 16:17:05 -07:00
Anu Engineer c9077a9f5a HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.
 2018-10-21 22:50:28 -07:00
Arpit Agarwal dc27408043 HDDS-612. Even after setting hdds.scm.chillmode.enabled to false, SCM allocateblock fails with ChillModePrecheck exception. Contributed by Hanisha Koneru. 2018-10-19 11:36:28 -07:00
Nandakumar 50715c0699 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar. 2018-10-17 17:45:35 +05:30
Márton Elek a9a63ae4a8 HDDS-527. Show SCM chill mode status in SCM UI. Contributed by Yiqun Lin. 2018-10-17 12:54:01 +02:00
Nandakumar 64a43c92c2 HDDS-656. Add logic for pipeline report and action processing in new pipeline code. Contributed by Lokesh Jain. 2018-10-17 13:57:38 +05:30
Arpit Agarwal 2614078b21 HDDS-490. Improve om and scm start up options . Contributed by Namit Maheshwari. 2018-10-15 16:45:08 -07:00
Nandakumar 5c8e023ba3 HDDS-587. Add new classes for pipeline management. Contributed by Lokesh Jain. 2018-10-13 03:22:20 +05:30
Bharat Viswanadham 74db993a61 HDDS-609. On restart, SCM does not exit chill mode as it expects DNs to report containers in ALLOCATED state. Contributed by Hanisha Koneru. 2018-10-11 21:03:01 -07:00
Hanisha Koneru eb34b5f8af HDDS-601. On restart, SCM throws 'No such datanode' exception. 2018-10-11 13:20:32 -07:00
Nanda kumar 90552b1ea8 HDDS-608. Access denied exception for user hdfs while making getContainerWithPipeline call. Contributed by Mukul Kumar Singh. 2018-10-10 20:50:49 +05:30
Akira Ajisaka 6fa3feb577
HADOOP-15832. Upgrade BouncyCastle to 1.60. Contributed by Robert Kanter. 2018-10-10 10:16:57 +09:00
Bharat Viswanadham 6a39739316 HDDS-583. SCM returns zero as the return code, even when invalid options are passed. Contributed by Namit Maheshwari. 2018-10-09 15:21:19 -07:00
Márton Elek 600438bcde HDDS-602. Bump Ozone version to 0.4.0-SNAPSHOT after ozone-0.3 branch cut. 2018-10-09 19:04:25 +02:00
Ajay Kumar cdf5d58364 HDDS-571. Update SCM chill mode exit criteria to optionally wait for n datanodes. Contributed by Ajay Kumar. 2018-10-05 14:07:16 -07:00
Nanda kumar f362037527 HDDS-575. LoadExistingContainers should be moved to SCMContainerManager from ContainerStateManager. Contributed by Mukul Kumar Singh. 2018-10-05 16:47:01 +05:30
Nanda kumar 095c269620 HDDS-567. Rename Mapping to ContainerManager in SCM. Contributed by Nanda kumar. 2018-10-03 16:00:39 +05:30
Nanda kumar 7b374482d2 HDDS-526. Clean previous chill mode code from NodeManager. Contributed by Ajay Kumar. 2018-10-03 15:15:43 +05:30
Nanda kumar a39296260f HDDS-561. Move Node2ContainerMap and Node2PipelineMap to NodeManager. Contributed by Lokesh Jain. 2018-10-02 19:47:44 +05:30
Ajay Kumar cc80ac2315 HDDS-557. DeadNodeHandler should handle exception from removeContainerHandler api. Contributed by Ajay Kumar. 2018-10-01 11:21:26 -07:00
Nanda kumar f7ff8c051e HDDS-325. Add event watcher for delete blocks command. Contributed by Lokesh Jain. 2018-10-01 13:50:17 +05:30
Ajay Kumar 3c798c1e3c HDDS-448. Move NodeStat to NodeStatemanager from SCMNodeManager. Contributed by LiXin Ge. 2018-09-28 10:33:17 -07:00
Ajay Kumar 184544eff8 HDDS-401. Update storage statistics on dead node. Contributed by LiXin Ge. 2018-09-27 09:56:30 -07:00
Ajay Kumar 3928af983e HDDS-370. Add and implement following functions in SCMClientProtocolServer. Contributed by Ajay Kumar. 2018-09-25 13:08:44 -07:00