Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,966 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

461 Commits

Author SHA1 Message Date
Xiaoyu Yao 2778259b88 HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593) 
Closes PR #593
 2019-03-12 09:34:18 -07:00
Mukul Kumar Singh daf4660eeb HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh. 2019-03-12 20:02:19 +05:30
Márton Elek d17e31e062
HDDS-1214. Enable tracing for the datanode read/write path. Contributed by Elek, Marton. 
Closes #550.
 2019-03-12 11:34:56 +01:00
Ajay Yadav 599e926d46 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar. 
This closes (#581)
 2019-03-11 15:01:04 -07:00
Bharat Viswanadham 68aa2ccd89
HDDS-1240. Fix check style issues caused by HDDS-1196. 2019-03-09 21:50:46 -08:00
Bharat Viswanadham 341c076f05 HDDS-1196. Add a ReplicationStartTimer class. (#567) 2019-03-08 20:55:33 +05:30
Yiqun Lin fb851c9481 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan. 2019-03-08 16:59:41 +08:00
Ajay Yadav 064f38b3a5 HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547) 2019-03-07 14:41:52 -08:00
Hanisha Koneru bb12e81ec8
HDDS-1175. Serve read requests directly from RocksDB. (#557) 
HDDS-1175. Serve read requests directly from RocksDB.
 2019-03-06 19:44:55 -08:00
Vivek Ratnavel Subramanian a55fc36299 HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values. 2019-03-06 17:43:57 -08:00
Shashikant Banerjee 62e89dc275 HDDS-1184. Parallelization of write chunks in datanodes is broken. Contributed by Shashikant Banerjee. 2019-03-06 10:00:16 +05:30
Arpit Agarwal 1ad5bfc53f
HDDS-1188. Implement a skeleton patch for Recon server with initial set of interfaces. Contributed by Siddharth Wagle. 2019-03-05 15:54:29 -08:00
Elek, Márton 7f636b47b5 HDDS-919. Enable prometheus endpoints for Ozone datanodes (#502) 
Pull request from Elek, Márton. (#502)
 2019-03-05 12:04:57 -08:00
Hanisha Koneru 12402b7a74 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-05 09:24:22 -08:00
Shashikant Banerjee 5fcea54a50 HDDS-935. Avoid creating an already created container on a datanode in case of disk removal followed by datanode restart. Contributed by Shashikant Banerjee. 2019-03-05 22:09:46 +05:30
Anu Engineer 10b802b84b HDDS-1136 : Add metric counters to capture the RocksDB checkpointing statistics. 
Contributed by Aravindan Vijayan.
 2019-03-04 12:00:16 -08:00
Xiaoyu Yao eed8b794d8 HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao. (#545) 2019-03-04 10:37:26 -08:00
Hanisha Koneru b18c1c22ea
Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client." 
This reverts commit 8e1225991d.
 2019-03-04 16:58:59 +09:00
Ajay Yadav 9e53088493 Revert "HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contr…" (#544) 
* Revert "Fix checkstyle issue"

This reverts commit 78c8a3323e.

* Revert "HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao."

This reverts commit 6fef6fc5ee.
 2019-03-01 16:19:43 -08:00
Xiaoyu Yao 6fef6fc5ee HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao. 2019-03-01 16:15:20 -08:00
Ajay Kumar 4a3cddea70 HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. 2019-03-01 15:41:09 -08:00
Xiaoyu Yao eea40cb88a
HDDS-1204. Fix ClassNotFound issue with javax.xml.bind.DatatypeConverter used by DefaultProfile. Contributed by Xiaoyu Yao. 2019-03-01 10:49:39 -08:00
Shashikant Banerjee de1dae64f2 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee. 2019-03-01 23:30:28 +05:30
Hanisha Koneru 8e1225991d HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-01 00:27:39 -08:00
Bharat Viswanadham 77b23c816e
HDDS-1182. Pipeline Rule where at least one datanode is reported in the pipeline. 2019-02-28 21:41:36 -08:00
Ajay Kumar cbf82fabf0 HDDS-1061. DelegationToken: Add certificate serial id to Ozone Delegation Token Identifier. Contributed by Ajay Kumar. 2019-02-27 17:00:42 -08:00
Ajay Kumar fef2e2fc88 HDDS-1176. Allow persisting X509CertImpl to SCM certificate table. Contributed by Xiaoyu Yao. 2019-02-27 10:40:41 -08:00
Bharat Viswanadham 6c8c422beb
HDDS-1178. Healthy pipeline Chill Mode Rule. 
Closes #518
 2019-02-27 14:26:40 +01:00
Xiaoyu Yao 9e0f3d1c52 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar. 2019-02-26 06:02:10 -08:00
Anu Engineer 9de34d2990 HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol. 
Contributed by Elek, Marton.
 2019-02-25 13:50:37 -08:00
Anu Engineer 5c1f946071 HDDS-1145. Add optional web server to the Ozone freon test tool. 
Contributed by Elek, Marton.
 2019-02-24 15:04:25 -08:00
Anu Engineer d82014b70a HDDS-1149. Change the default ozone.client.checksum.type. 
Contributed by Bharat Viswanadham.
 2019-02-24 13:22:36 -08:00
Anu Engineer 8387bbdfb5 HDDS-1154. Add tracing to the client side of StorageContainerLocationProtocol and OzoneManagerProtocol. 
Contributed by Elek, Marton.
 2019-02-24 13:13:37 -08:00
Bharat Viswanadham 106bdc6c04
HDDS-1120. Add a config to disable checksum verification during read. 2019-02-23 21:37:55 -08:00
Bharat Viswanadham d33f0666f6
HDDS-1141. Update DBCheckpointSnapshot to DBCheckpoint. 
* HDDS-1141.Update DBCheckpointSnapshot to DBCheckpoint.

* fix test failures in TestOzoneConfigurationFields
 2019-02-21 13:29:10 -08:00
Arpit Agarwal 676a9cb888 HDDS-1053. Generate RaftGroupId from OMServiceID. Contributed by Aravindan Vijayan. 2019-02-20 12:57:49 -08:00
Xiaoyu Yao 1374f8f548 HDDS-1060. Add API to get OM certificate from SCM CA. Contributed by Ajay Kumar. 2019-02-20 11:35:49 -08:00
avijayanhwx e8d7e3b4e6 HDDS-1139 : Fix findbugs issues caused by HDDS-1085. Contributed by Aravindan Vijayan. 2019-02-19 16:32:47 -08:00
Anu Engineer 02d04bd107 HDDS-1121. Key read failure when data is written parallel in to Ozone. 
Contributed by Bharat Viswanadham.
 2019-02-19 11:07:17 -08:00
Xiaoyu Yao cf1a66d0da HDDS-1101. SCM CA: Write Certificate information to SCM Metadata. Contributed by Anu Engineer. 2019-02-19 10:42:56 -08:00
Anu Engineer 588b4c4d78 HDDS-1085. Create an OM API to serve snapshots to Recon server. 
Contributed by Aravindan Vijayan.
 2019-02-18 22:35:36 -08:00
Anu Engineer 217bdbd940 HDDS-1116.Add java profiler servlet to the Ozone web servers. 
Contributed by Elek, Marton.
 2019-02-15 11:32:06 -08:00
Shashikant Banerjee de934ba2dc HDDS-1076. TestSCMNodeManager crashed the jvm. Contributed by Lokesh Jain. 2019-02-15 21:13:30 +05:30
Anu Engineer 75e15cc0c4 HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects. 
Contributed by Elek, Marton.
 2019-02-14 23:33:25 -08:00
Anu Engineer 0395f22145 HDDS-1068. Improve the error propagation for ozone sh. 
Contributed by Elek, Marton.
 2019-02-14 22:51:46 -08:00
Bharat Viswanadham 917ac9f108 HDDS-972. Add support for configuring multiple OMs. Contributed by Hanisha Koneru. 2019-02-12 22:07:27 -08:00
Nanda kumar a536eb5c41 HDDS-360. Use RocksDBStore and TableStore for SCM Metadata. 
Contributed by Anu Engineer.
 2019-02-12 14:26:19 +05:30
Ajay Kumar d48e61dd36 HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar 2019-02-11 18:59:47 -08:00
Márton Elek e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Shashikant Banerjee 965d26c9c7 HDDS-1026. Reads should fail over to alternate replica. Contributed by Shashikant Banerjee. 2019-02-10 10:53:16 +05:30
Nanda kumar 0c1bc4dcee HDDS-981. Block allocation should involve pipeline selection and then container selection. 
Contributed by Lokesh Jain.
 2019-02-08 15:44:41 +05:30
Márton Elek a65aca2fef HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton. 2019-02-07 17:02:03 +01:00
Márton Elek 9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Yiqun Lin 3efa168e1f HDDS-1029. Allow option for force in DeleteContainerCommand. Contributed by Bharat Viswanadham. 2019-02-05 10:51:52 +08:00
Ajay Kumar 9a19d6d48b HDDS-1039. OzoneManager fails to connect with secure SCM. Contributed by Ajay Kumar 2019-02-04 16:30:07 -08:00
Márton Elek 033d97abac HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham. 2019-01-31 14:51:31 +01:00
Nanda kumar 45c4cfe790 HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version. 
Contributed by Doroszlai, Attila.
 2019-01-25 12:16:29 +05:30
Xiaoyu Yao e321b91cb5 HDDS-980. Adding getOMCertificate in SCMSecurityProtocol. Contributed by Ajay Kumar. 2019-01-23 19:32:03 -08:00
Bharat Viswanadham 0ef54f72a6 HDDS-992. ozone-default.xml has invalid text from a stale merge. Contributed by Dinesh Chitlangia. 2019-01-22 13:37:05 -08:00
Xiaoyu Yao 2aaaf12f9e HDDS-943. Add block token validation in HddsDispatcher/XceiverServer. Contributed by Ajay Kumar. 2019-01-18 12:11:34 -08:00
Ajay Kumar 01a7f9ed35 HDDS-597. Ratis: Support secure gRPC endpoint with mTLS for Ratis. Contributed by Ajay Kumar. 2019-01-17 10:30:44 -08:00
Xiaoyu Yao 140565f4db HDDS-967. Fix failures in TestOzoneConfigurationFields. Contributed by Ajay Kumar. 2019-01-16 16:44:33 -07:00
Ajay Kumar 0e16cf1973 HDDS-970. Fix classnotfound error for bouncy castle classes in OM,SCM init. Contributed by Ajay Kumar. 2019-01-16 16:40:05 -07:00
Anu Engineer a5d0fcf704 HDDS-963. Fix failure in TestOzoneShell due to null check in SecurityConfig. 
Contributed by Ajay Kumar.
 2019-01-16 16:18:39 -07:00
Anu Engineer 1d5734e341 HDDS-938. Add Client APIs for using S3 Auth interface. 
Contributed by  Dinesh Chitlangia.
 2019-01-16 16:18:39 -07:00
Anu Engineer 30bfc9cbd0 HDDS-955. SCM CA: Add CA to SCM. 
Contributed by Anu Engineer.
 2019-01-16 13:43:31 -07:00
Xiaoyu Yao 924bea9730 HDDS-102. SCM CA: SCM CA server signs certificate for approved CSR. Contributed by Anu Engineer. 2019-01-16 13:43:30 -07:00
Ajay Kumar 50c4045fde HDDS-805. Block token: Client api changes for block token. Contributed by Ajay Kumar. 2019-01-16 13:38:27 -07:00
Xiaoyu Yao 2b115222cd HDDS-929. Remove ozone.max.key.len property. Contributed by Ajay Kumar. 2019-01-16 01:42:08 -07:00
Ajay Kumar f894d86b2f HDDS-115. GRPC: Support secure gRPC endpoint with mTLS. Contributed by Xiaoyu Yao. 2019-01-16 01:42:08 -07:00
Ajay Kumar 6d522dc05d HDDS-804. Block token: Add secret token manager. Contributed by Ajay Kumar. 2019-01-16 01:22:43 -07:00
Anu Engineer 0c8829a9a1 HDDS-696. Bootstrap genesis SCM(CA) with self-signed certificate. 
Contributed by Anu Engineer.
 2019-01-16 01:16:25 -07:00
Xiaoyu Yao 7e2770699c HDDS-9. Add GRPC protocol interceptors for Ozone Block Token. Contributed by Xiaoyu Yao. 2019-01-16 01:10:08 -07:00
Ajay Kumar bb4a26ca32 HDDS-8. Add OzoneManager Delegation Token support. Contributed by Ajay Kumar. 2019-01-16 01:06:59 -07:00
Xiaoyu Yao 6d6b1a00c2 HDDS-836. Add TokenIdentifier Ozone for delegation token and block token. Contributed by Ajay Kumar. 2019-01-15 23:01:04 -07:00
Anu Engineer 6ad794b1b6 HDDS-778. Add an interface for CA and Clients for Certificate operations 
Contributed by Anu Engineer.
 2019-01-15 23:01:03 -07:00
Ajay Kumar 8b8a3f5435 HDDS-760. Add asf license to TestCertificateSignRequest. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Ajay Kumar 33c274ea2e HDDS-103. SCM CA: Add new security protocol for SCM to expose security related functions. Contributed by Ajay Kumar. 2019-01-15 23:01:03 -07:00
Anu Engineer c260c19d18 HDDS-101. SCM CA: generate CSR for SCM CA clients. 
Contributed by Xiaoyu Yao.
 2019-01-15 23:01:03 -07:00
Xiaoyu Yao e89c35a50b HDDS-704. Fix the Dependency convergence issue on HDDS-4. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar 8d7c5f4ee7 HDDS-588. SelfSignedCertificate#generateCertificate should sign the certificate the configured security provider. Contributed by Xiaoyu Yao. 2019-01-15 23:01:03 -07:00
Ajay Kumar 9920506b3d HDDS-547. Fix secure docker and configs. Contributed by Xiaoyu Yao. 2019-01-15 23:01:02 -07:00
Ajay Kumar 2d269440b0 HDDS-548. Create a Self-Signed Certificate. Contributed by Anu Engineer. 2019-01-15 22:54:51 -07:00
Ajay Kumar 0b034b7005 HDDS-546. Resolve bouncy castle dependency for hadoop-hdds-common. Contributed by Ajay Kumar. 2019-01-15 22:54:51 -07:00
Xiaoyu Yao 570b503e3e Fix merge conflicts 2019-01-15 22:54:50 -07:00
Xiaoyu Yao faf53f8262 HDDS-100. SCM CA: generate public/private key pair for SCM/OM/DNs. Contributed by Ajay Kumar. 2019-01-15 22:53:56 -07:00
Xiaoyu Yao e47135d9d9 HDDS-70. Fix config names for secure ksm and scm. Contributed by Ajay Kumar. 2019-01-15 22:53:55 -07:00
Xiaoyu Yao bfa4929fdb Revert "Bad merge with 996a627b289947af3894bf83e7b63ec702a665cd" 
This reverts commit 996a627b289947af3894bf83e7b63ec702a665cd.
 2019-01-15 22:40:08 -07:00
Xiaoyu Yao 8e6743eb90 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Xiaoyu Yao d3920ecbb7 HDDS-5. Enable OzoneManager kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:40:08 -07:00
Xiaoyu Yao ff61931f91 HDDS-6. Enable SCM kerberos auth. Contributed by Ajay Kumar. 2019-01-15 22:16:58 -07:00
Márton Elek 32cf0412ce HDDS-965. Ozone: checkstyle improvements and code quality scripts. Contributed by Elek, Marton. 2019-01-08 22:27:19 +01:00
Márton Elek d14c56d150 HDDS-916. MultipartUpload: Complete Multipart upload request. Contributed by Bharat Viswanadham. 2019-01-07 11:27:36 +01:00
Anu Engineer 999da98d67 HDDS-915. Submit client request to OM Ratis server. 
Contributed by Hanisha Koneru.
 2019-01-04 15:25:26 -08:00
Anu Engineer 0cb3316cec HDDS-482. NullPointer exception thrown on console when cli operation failed. 
Contributed by Nandakumar.
 2019-01-02 11:30:54 -08:00
Arpit Agarwal cf571133b8 HDDS-893. pipeline status is ALLOCATED in scmcli listPipelines command. Contributed by Lokesh Jain. 2018-12-19 13:42:06 +05:30
Shashikant Banerjee b51e9e4317 HDDS-912. Update ozone to latest ratis snapshot build (0.4.0-3b0be02-SNAPSHOT). Contributed by Lokesh Jain. 2018-12-18 18:21:17 +05:30
Xiaoyu Yao 94b368f29f HDDS-99. Adding SCM Audit log. Contributed by Dinesh Chitlangia. 2018-12-17 15:40:22 -08:00
Márton Elek 17a8708039 HDDS-879. MultipartUpload: Add InitiateMultipartUpload in ozone. Contributed by Bharat Viswanadham. 2018-12-10 10:05:38 +01:00
Jitendra Pandey 1afba83f2c HDDS-870. Avoid creating block sized buffer in ChunkGroupOutputStream. Contributed by Shashikant Banerjee. 2018-12-08 09:23:10 -08:00
Anu Engineer 5e773efd78 HDDS-909 : Default implementation for Ozone acls. 
Contributed by Ajay Kumar.
 2018-12-07 23:20:19 -08:00
Hanisha Koneru 019836b113 HDDS-858. Start a Standalone Ratis Server on OM 2018-12-06 13:33:52 -08:00
Ajay Kumar 8d882c3786 HDDS-880. Create api for ACL handling in Ozone. (Contributed by Ajay Kumar) 2018-12-06 13:27:28 -08:00
Bharat Viswanadham 343aaea2d1 HDDS-864. Use strongly typed codec implementations for the tables of the OmMetadataManager. Contributed by Elek Marton. 2018-12-06 11:39:59 -08:00
Bharat Viswanadham ef3b03b75a HDDS-885. Fix test failures due to ChecksumData. Contributed by Hanisha Koneru. 2018-12-03 10:52:49 -08:00
Shashikant Banerjee 8f3e12ff07 HDDS-882. Provide a config to optionally turn on/off the sync flag during chunk writes. Contributed by Shashikant Banerjee. 2018-12-02 08:06:24 +05:30
Shashikant Banerjee 5a3c7714c4 HDDS-887. Add DispatcherContext info to Dispatcher from containerStateMachine. Contributed by Shashikant Banerjee. 2018-12-02 08:00:35 +05:30
Bharat Viswanadham d15dc43659 HDDS-748. Use strongly typed metadata Table implementation. Contributed by Elek Marton. 2018-12-01 16:52:23 -08:00
Shashikant Banerjee 5e102f9aa5 HDDS-850. ReadStateMachineData hits OverlappingFileLockException in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-11-29 22:20:08 +05:30
Ajay Kumar 184cced513 HDDS-808. Simplify OMAction and DNAction classes used for AuditLogging. Contributed by Dinesh Chitlangia. 2018-11-29 08:35:20 -08:00
Ajay Kumar b71cc7f33e HDDS-642. Add chill mode exit condition for pipeline availability. Contributed by Yiqun Lin. 2018-11-28 17:47:57 -08:00
Hanisha Koneru 64a4b6b08b HDDS-284. Checksum for ChunksData. 2018-11-28 10:53:12 -08:00
Anu Engineer 34a914be03 HDDS-846. Exports ozone metrics to prometheus. 
Contributed by Elek, Marton.
 2018-11-27 15:57:24 -08:00
Nanda kumar 099e723406 HDDS-868. Handle quasi closed container replicas in SCM. 
Contributed by Nanda kumar.
 2018-11-27 13:01:22 +05:30
Mukul Kumar Singh f207e30142 HDDS-814. dfs.ratis.leader.election.minimum.timeout.duration should not be read by client. Contributed by chencan. 2018-11-22 10:46:10 +05:30
Ajay Kumar 8d2789c5eb HDDS-795. RocksDb specific classes leak from DBStore/Table interfaces. Contributed by Márton Elek. 2018-11-21 12:25:41 -08:00
Bharat Viswanadham c4d0ef6946 HDDS-816. Create OM metrics for bucket, volume, keys. Contributed by Bharat Viswanadham. 2018-11-21 10:35:39 -08:00
Shashikant Banerjee 14e1a0a3d6 HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-21 10:51:50 +05:30
Shashikant Banerjee 1734ace35f Revert "HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee." 
This reverts commit c74783036d.
 2018-11-21 01:07:38 +05:30
Shashikant Banerjee c74783036d HDDS-835. Use storageSize instead of Long for buffer size configs in Ozone Client. Contributed by Shashikant Banerjee. 2018-11-20 23:01:02 +05:30
Mukul Kumar Singh b5d7b292c9 HDDS-718. Introduce new SCM Commands to list and close Pipelines. Contributed by Lokesh Jain. 2018-11-19 22:59:30 +05:30
Nanda kumar 5a7ca6ac3e HDDS-837. Persist originNodeId as part of .container file in datanode. 
Contributed by Nanda kumar.
 2018-11-19 14:39:55 +05:30
Shashikant Banerjee 10cf5773ba HDDS-845. Create a new raftClient instance for every watch request for Ratis. Contributed by Shashikant Banerjee. 2018-11-19 14:38:51 +05:30
Mukul Kumar Singh cfb915f3df HDDS-844. Add logic for pipeline teardown after timeout. Contributed by Lokesh Jain. 2018-11-19 12:11:05 +05:30
Xiaoyu Yao 29374999b6 HDDS-120. Adding HDDS datanode Audit Log. Contributed by Dinesh Chitlangia. 2018-11-16 14:59:59 -08:00
Nanda kumar c4d9640028 HDDS-801. Quasi close the container when close is not executed via Ratis. 
Contributed by Nanda kumar.
 2018-11-16 23:11:33 +05:30
Anu Engineer a16aa2f60b HDDS-825. Code cleanup based on messages from ErrorProne. 
Contributed by Anu Engineer.
 2018-11-15 17:36:09 -08:00
Giovanni Matteo Fumarola fcd94eeab8 HDDS-842. [JDK11] Fix Javadoc errors in hadoop-hdds-common module. Contributed by Dinesh Chitlangia. 2018-11-15 14:59:31 -08:00
Márton Elek 2718f7534d HDDS-828. Fix deprecation log generated by getting value of the setting ozone.scm.heartbeat.rpc-timeout. Contributed by Yiqun Lin. 2018-11-15 14:04:55 +01:00
Márton Elek fb1e8bb0cc HDDS-528. add cli command to checkChill mode status and exit chill mode. Contributed by chencan. 2018-11-15 13:59:56 +01:00
Márton Elek b12e69475b HDDS-658. Implement s3 bucket list backend call and use it from rest endpoint. Contributed by Bharat Viswanadham. 2018-11-15 12:22:16 +01:00
Shashikant Banerjee a948281706 HDDS-834. Datanode goes OOM based because of segment size. Contributed by Mukul Kumar Singh. 2018-11-14 15:53:22 +05:30
Shashikant Banerjee 671fd6524b HDDS-675. Add blocking buffer and use watchApi for flush/close in OzoneClient. Contributed by Shashikant Banerjee. 2018-11-13 23:39:14 +05:30
Jitendra Pandey f944f33832 HDDS-709. Modify Close Container handling sequence on datanodes. Contributed by Shashikant Banerjee. 2018-11-12 14:08:39 -08:00
Nanda kumar 18fe65d756 HDDS-576. Move ContainerWithPipeline creation to RPC endpoint. 
Contributed by Nanda kumar.
 2018-11-12 23:33:11 +05:30
Nanda kumar 9fe50b4991 HDDS-733. Create container if not exist, as part of chunk write. 
Contributed by Lokesh Jain.
 2018-11-10 00:08:17 +05:30
Shashikant Banerjee 31614bcc7c HDDS-806. Update Ratis to latest snapshot version in ozone. Contributed by Tsz Wo Nicholas Sze and Mukul Kumar Singh. 2018-11-09 00:05:45 +05:30
Nanda kumar c80f753b0e HDDS-737. Introduce Incremental Container Report. 
Contributed by Nanda kumar.
 2018-11-08 18:33:38 +05:30
Akira Ajisaka 49412a1289
HDDS-811. [JDK 11] Fix compilation failures with jdk 11. Contributed by Lokesh Jain. 2018-11-06 21:26:43 +09:00
Arpit Agarwal 5ddefdd507 HDDS-794. Add configs to set StateMachineData write timeout in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-11-05 10:41:28 -08:00
Yiqun Lin d16d5f730e HDDS-751. Replace usage of Guava Optional with Java Optional. 2018-11-02 10:50:32 +08:00
Shashikant Banerjee b13c56742a HDDS-697. update and validate the BCSID for PutSmallFile/GetSmallFile command. Contributed by Shashikant Banerjee. 2018-11-01 10:21:39 +05:30
Arpit Agarwal 08bb0362e0 HDDS-759. Create config settings for SCM and OM DB directories. Contributed by Arpit Agarwal. 2018-10-31 11:23:15 -07:00
Nanda kumar e4f22b08e0 HDDS-755. ContainerInfo and ContainerReplica protobuf changes. 
Contributed by Nanda kumar.
 2018-10-31 10:29:35 +05:30
Shashikant Banerjee 7757331dbc HDDS-749. Restructure BlockId class in Ozone. Contributed by Shashikant Banerjee. 2018-10-30 14:15:27 +05:30
Arpit Agarwal 496f0ffe90 HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal. 2018-10-29 17:14:18 -07:00
Arpit Agarwal a58048e8d5 HDDS-727. ozone.log is not getting created in logs directory. Contributed by Nilotpal Nandi. 2018-10-29 09:35:18 -07:00
Nanda kumar 2fa01f823c HDDS-744. Fix ASF license warning in PipelineNotFoundException class. 
Contributed by Lokesh Jain.
 2018-10-27 16:51:08 +05:30
Nanda kumar dce4ebe814 HDDS-694. Plugin new Pipeline management code in SCM. 
Contributed by Lokesh Jain.
 2018-10-26 17:55:04 +05:30
Hanisha Koneru 38a65e3b7c HDDS-682. Unified o3 address parsing for ozonen sh. Contributed by Elek, Marton. 2018-10-25 17:04:16 -07:00
Arpit Agarwal 244afaba4a HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal. 2018-10-24 16:17:05 -07:00
Shashikant Banerjee b61846392e HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee. 2018-10-23 16:52:17 +05:30
Anu Engineer e3cca12048 HDDS-676. Enable Read from open Containers via Standalone Protocol. 
Contributed by Shashikant Banerjee.
 2018-10-22 15:57:01 -07:00
Mukul Kumar Singh 82919a1e7a HDDS-638. Enable ratis snapshots for HDDS datanodes. Contributed by Mukul Kumar Singh. 2018-10-22 19:59:35 +05:30
Nanda kumar c696419f3e Revert "HDDS-705. OS3Exception resource name should be the actual resource name." 
This reverts commit 977c6f6470.

There was spurious edit in this commit.
 2018-10-22 15:37:40 +05:30
Anu Engineer c9077a9f5a HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.
 2018-10-21 22:50:28 -07:00
Anu Engineer 977c6f6470 HDDS-705. OS3Exception resource name should be the actual resource name. 
Contributed by Bharat Viswanadham.
 2018-10-21 22:49:16 -07:00
Jitendra Pandey d93d515af5 HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:19:17 -07:00
Nandakumar 50715c0699 HDDS-662. Introduce ContainerReplicaState in StorageContainerManager. Contributed by Nanda kumar. 2018-10-17 17:45:35 +05:30
Nandakumar 64a43c92c2 HDDS-656. Add logic for pipeline report and action processing in new pipeline code. Contributed by Lokesh Jain. 2018-10-17 13:57:38 +05:30
Arpit Agarwal 5f0b43fa93 HDDS-378. Remove dependencies between hdds/ozone and hdfs proto files. Contributed by Elek, Marton. 2018-10-16 10:40:00 -07:00
Jitendra Pandey 603649d3a9 HDDS-579. ContainerStateMachine should fail subsequent transactions per container in case one fails. Contributed by Shashikant Banerjee. 2018-10-13 19:15:01 -07:00
Jitendra Pandey 5209c7503b HDDS-603. Add BlockCommitSequenceId field per Container and expose it in Container Reports. Contributed by Shashikant Banerjee. 2018-10-13 12:15:42 -07:00
Jitendra Pandey 22f37af935 Revert "HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee." 
This reverts commit 0473b68000.
 2018-10-13 12:14:39 -07:00
Jitendra Pandey 0473b68000 HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee. 2018-10-13 12:12:08 -07:00
Anu Engineer 9227f3d224 HDDS-653. TestMetadataStore#testIterator fails on Windows. 
Contributed by Yiqun Lin.
 2018-10-13 10:41:06 -07:00
Nandakumar 5c8e023ba3 HDDS-587. Add new classes for pipeline management. Contributed by Lokesh Jain. 2018-10-13 03:22:20 +05:30
Arpit Agarwal 3a684a2b23 HDDS-644. Rename dfs.container.ratis.num.container.op.threads. Contributed by Bharat Viswanadham. 2018-10-12 14:44:51 -07:00
Anu Engineer 02e1ef5e07 HDDS-624.PutBlock fails with Unexpected Storage Container Exception. 
Contributed by  Arpit Agarwal.
 2018-10-12 14:22:46 -07:00
Arpit Agarwal 50552479c4 HDDS-636. Turn off ozone.metastore.rocksdb.statistics for now. Contributed by Arpit Agarwal. 2018-10-11 17:21:27 -07:00
Jitendra Pandey 0bc6d0484a HDDS-550. Serialize ApplyTransaction calls per Container in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-10-11 17:12:47 -07:00
Arpit Agarwal 2bd000c851 HDDS-619. hdds.db.profile should not be tagged as a required setting & should default to DISK. Contributed by Dinesh Chitlangia. 2018-10-10 17:37:20 -07:00
Márton Elek 600438bcde HDDS-602. Bump Ozone version to 0.4.0-SNAPSHOT after ozone-0.3 branch cut. 2018-10-09 19:04:25 +02:00
Nanda kumar 9e9915ddab HDDS-599. Fix TestOzoneConfiguration TestOzoneConfigurationFields. Contributed by Sandeep Nemuri. 2018-10-09 18:23:05 +05:30
Nanda kumar 7367ff333b HDDS-450. Generate BlockCommitSequenceId in ContainerStateMachine for every commit operation in Ratis. Contributed by Shashikant Banerjee. 2018-10-09 18:07:01 +05:30
Ajay Kumar cdf5d58364 HDDS-571. Update SCM chill mode exit criteria to optionally wait for n datanodes. Contributed by Ajay Kumar. 2018-10-05 14:07:16 -07:00
Bharat Viswanadham 2a07617f85 HDDS-354. VolumeInfo.getScmUsed throws NPE. Contributed by Hanisha Koneru. 2018-10-04 21:46:17 -07:00
Anu Engineer e6b77ad65f HDDS-572. Support S3 buckets as first class objects in Ozone Manager - 1. 
Contributed by Anu Engineer.
 2018-10-04 21:40:13 -07:00
Bharat Viswanadham 4eff629ab3 HDDS-525. Support virtual-hosted style URLs. Contributed by Bharat Viswanadham 2018-10-01 13:16:08 -07:00
Nanda kumar f7ff8c051e HDDS-325. Add event watcher for delete blocks command. Contributed by Lokesh Jain. 2018-10-01 13:50:17 +05:30
Anu Engineer 0da03f8b14 HDDS-391. Simplify Audit Framework to make audit logging easier to use. 
Contributed by Dinesh Chitlangia.
 2018-09-28 14:00:24 -07:00
Nanda kumar 3f6195045e HDDS-554. In XceiverClientSpi, implement sendCommand(..) using sendCommandAsync(..). Contributed by Tsz Wo Nicholas Sze. 2018-09-26 09:16:23 +05:30
Ajay Kumar 3928af983e HDDS-370. Add and implement following functions in SCMClientProtocolServer. Contributed by Ajay Kumar. 2018-09-25 13:08:44 -07:00
Bharat Viswanadham 9c3fbbc4f6 HDDS-444. Add rest service to the s3gateway. Contributed by Elek Marton. 2018-09-24 22:10:16 -07:00
Bharat Viswanadham 62f817d32e HDDS-447. Separate ozone-dist and hadoop-dist projects with real classpath separation. Contributed by Elek Marton. 2018-09-24 10:10:11 -07:00
Anu Engineer 096a716080 HDDS-394. Rename *Key Apis in DatanodeContainerProtocol to *Block apis. 
Contributed Dinesh Chitlangia.
 2018-09-20 11:51:49 -07:00
Nanda kumar c0956ee2a8 HDDS-476. Add Pipeline reports to make pipeline active on SCM restart. 
Contributed by Mukul Kumar Singh.
 2018-09-19 18:52:08 +05:30
Xiaoyu Yao e435e12f1f HDDS-506. Fields in AllocateScmBlockResponseProto should be optional. Contributed by Arpit Agarwal. 2018-09-18 23:21:29 -07:00
Xiaoyu Yao 295cce39ed HDDS-366. Update functions impacted by SCM chill mode in StorageContainerLocationProtocol. Contributed by Ajay Kumar. 2018-09-18 13:32:48 -07:00
Anu Engineer f938925bde HDDS-464. Fix TestCloseContainerHandlingByClient. 
Contributed by Lokesh Jain.
 2018-09-18 10:45:51 -07:00
Nanda kumar 846936698b HDDS-399. Persist open pipeline information across SCM restart. Contributed by Mukul Kumar Singh. 2018-09-17 21:51:54 +05:30
Xiaoyu Yao 95231f1749 HDDS-362. Modify functions impacted by SCM chill mode in ScmBlockLocationProtocol. Contributed by Ajay Kumar. 2018-09-16 17:55:46 -07:00
Nanda kumar 87e2c0f425 HDDS-465. Suppress group mapping lookup warnings for ozone. Contributed by Xiaoyu Yao. 2018-09-15 23:14:57 +05:30
Nanda kumar c9fa081897 HDDS-462. Optimize ContainerStateMap#getMatchingContainerIDs in SCM. Contributed by Nanda kumar. 2018-09-15 23:11:39 +05:30
Arpit Agarwal a12f12f1af HDDS-438. 'ozone oz' should print usage when command or sub-command is missing. Contributed by Dinesh Chitlangia. 2018-09-14 07:02:28 -07:00
Hanisha Koneru 291dcf2247 HDDS-423. Introduce an ozone specific log4j.properties. Contributed by Elek, Marton. 2018-09-13 16:37:44 -07:00
Mukul Kumar Singh c9e0b69ab3 HDDS-233. Update ozone to latest ratis snapshot build(0.3.0-50588bd-SNAPSHOT). Contributed by Shashikant Banerjee. 2018-09-13 19:01:07 +05:30
Anu Engineer c18eb97801 HDDS-395. TestOzoneRestWithMiniCluster fails with "Unable to read ROCKDB config". 
Contributed by Dinesh Chitlangia.
 2018-09-12 14:19:01 -07:00
Anu Engineer 64c7a12b57 HDDS-436. Allow SCM chill mode to be disabled by configuration. 
Contributed by Ajay Kumar.
 2018-09-12 11:40:24 -07:00
Anu Engineer 6e2129cf4e HDDS-428. OzoneManager lock optimization. 
Contributed by Nanda Kumar.
 2018-09-12 10:38:36 -07:00