186 Commits

Author SHA1 Message Date
Xiaoyu Yao
8d1c218428
HDDS-1299. Support TokenIssuer interface for running jobs with OzoneFileSystem. Contributed by Xiaoyu Yao.
This closes #627.

(cherry picked from commit 6a34c9bb29dc3e0370b63b63f26229e3dc90cc13)
2019-03-23 09:36:56 -07:00
Xiaoyu Yao
9b989c49c9 HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading from stale pipeline. Contributed by Xiaoyu Yao. (#639)
(cherry picked from commit dea6f2a065271d2e37eda05bbcb98fb1bb5ed2a6)
2019-03-22 15:09:22 -07:00
Arpit Agarwal
6c009a3d04
HDDS-1321. TestOzoneManagerHttpServer depends on hard-coded port numbers. Contributed by Arpit Agarwal. (#633)
Change-Id: I9656af4a7f41812da9d125c10ae0e8daf3dcf7f5
(cherry picked from commit 2828f8c339d7a03bd2bedf99c7700a3bbeec3a34)
2019-03-21 14:21:05 -07:00
Hanisha Koneru
598c16c1fc
HDDS-1072. Implement RetryProxy and FailoverProxy for OM client.
(cherry picked from commit 12402b7a74a9069230ebfdc9e5c2ee0e28a23990)
2019-03-20 17:53:06 -07:00
Ajay Yadav
d2e115d8c8 HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601)
(cherry picked from commit f10d49332522beca7cb7342e68b2acdbe4c974f8)
2019-03-18 23:11:36 -07:00
Xiaoyu Yao
6fd95b8595 HDDS-1296. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#616)
(cherry picked from commit 66a104bc57242b2817e8c1f67cb6776e70cf5c47)
2019-03-18 09:39:05 -07:00
Xiaoyu Yao
3ca1024eec HDDS-1138. Ozone Client should avoid talking to SCM directly. Contributed by Xiaoyu Yao and Mukul Kumar Singh.
Closes #585 

(cherry picked from commit d1afa038044f6aaec43551571caae567b2bf4139)
2019-03-15 10:42:09 -07:00
Vivek Ratnavel Subramanian
64726dd8fe
HDDS-761. Create S3 subcommand to run S3 related operations.
Closes #579

(cherry picked from commit 90015084850298803d8e165a0f6f9719ba724ea3)
2019-03-14 21:19:48 -07:00
Vivek Ratnavel Subramanian
c00e704496
HDDS-1265. ozone sh s3 getsecret throws Null Pointer Exception for unsecured clusters.
Closes #611

(cherry picked from commit 2627dad333eb88011b854fe163d70e79163e33f8)
2019-03-14 17:22:21 -07:00
Ajay Yadav
0f4ae392d7 HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598)
(cherry picked from commit 4fa009989bd641388aeb9b1bd9c90e9ddbfe84c0)
2019-03-13 12:02:17 -07:00
Ajay Yadav
15b6e38ddf
HDDS-1043. Enable token based authentication for S3 api
Closes #561

(cherry picked from commit dcb0de848d8388fcee425847e52f6361790225c6)
2019-03-12 18:54:59 +01:00
Mukul Kumar Singh
97fd3025a4 HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh.
(cherry picked from commit daf4660eebfa9267e7082f7c9a14c3dd5d6d5d46)
2019-03-12 20:03:36 +05:30
Lokesh Jain
60b5e178e3 HDDS-1220. KeyManager#openKey should release the bucket lock before doing an allocateBlock. Contributed by Lokesh Jain.
(cherry picked from commit ebb5fa115b842a3bcb747e881ca2a8fe4c716d07)
2019-03-11 14:56:51 +05:30
Márton Elek
9cd3fe49cf
HDDS-1113. Remove default dependencies from hadoop-ozone project. Contributed by Elek, Marton.
Closes #490

(cherry picked from commit 45f976fdcf1d8cbcdcbcc4783c9f13fddae1c429)
2019-03-07 14:44:34 +01:00
Ajay Yadav
c3986dc53b
HDDS-1156. testDelegationToken is failing in TestSecureOzoneCluster. Contributed by Ajay Kumar.
(cherry picked from commit 353b90d59e3583a73a76db550d47aa86b3f36d5d)
2019-03-05 08:39:45 -08:00
Hanisha Koneru
b18c1c22ea
Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client."
This reverts commit 8e1225991d8da7d6801fc3753319139873f23bc9.
2019-03-04 16:58:59 +09:00
Ajay Kumar
4a3cddea70 HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. 2019-03-01 15:41:09 -08:00
Shashikant Banerjee
de1dae64f2 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee. 2019-03-01 23:30:28 +05:30
Hanisha Koneru
8e1225991d HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-01 00:27:39 -08:00
Ajay Kumar
cbf82fabf0 HDDS-1061. DelegationToken: Add certificate serial id to Ozone Delegation Token Identifier. Contributed by Ajay Kumar. 2019-02-27 17:00:42 -08:00
Xiaoyu Yao
9e0f3d1c52 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar. 2019-02-26 06:02:10 -08:00
Anu Engineer
9de34d2990 HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol.
Contributed by Elek, Marton.
2019-02-25 13:50:37 -08:00
Anu Engineer
8387bbdfb5 HDDS-1154. Add tracing to the client side of StorageContainerLocationProtocol and OzoneManagerProtocol.
Contributed by Elek, Marton.
2019-02-24 13:13:37 -08:00
Bharat Viswanadham
106bdc6c04
HDDS-1120. Add a config to disable checksum verification during read. 2019-02-23 21:37:55 -08:00
Anu Engineer
c1e5b19212 HDDS-1147. Fix failing unit tests in TestOzoneManager.
Contributed by Nandakumar.
2019-02-23 20:56:24 -08:00
Bharat Viswanadham
d33f0666f6
HDDS-1141. Update DBCheckpointSnapshot to DBCheckpoint.
* HDDS-1141.Update DBCheckpointSnapshot to DBCheckpoint.

* fix test failures in TestOzoneConfigurationFields
2019-02-21 13:29:10 -08:00
Márton Elek
7c802c42dc
HDDS-1129. Fix findbug/checkstyle errors hdds projects. Contributed by Elek, Marton. 2019-02-21 14:04:18 +01:00
Arpit Agarwal
676a9cb888 HDDS-1053. Generate RaftGroupId from OMServiceID. Contributed by Aravindan Vijayan. 2019-02-20 12:57:49 -08:00
avijayanhwx
e8d7e3b4e6 HDDS-1139 : Fix findbugs issues caused by HDDS-1085. Contributed by Aravindan Vijayan. 2019-02-19 16:32:47 -08:00
Anu Engineer
588b4c4d78 HDDS-1085. Create an OM API to serve snapshots to Recon server.
Contributed by Aravindan Vijayan.
2019-02-18 22:35:36 -08:00
Yiqun Lin
67af509097 HDDS-1122. Fix TestOzoneManagerRatisServer#testSubmitRatisRequest unit test failure. Contributed by Yiqun Lin. 2019-02-19 11:29:52 +08:00
Anu Engineer
7ea91494ce HDDS-1041. Support TDE(Transparent Data Encryption) for Ozone.
Contributed by Xiaoyu Yao.
2019-02-16 10:23:10 -08:00
Elek, Márton
afe126d71f HDDS-1114. Fix findbugs/checkstyle/accepteance errors in Ozone. Contributed by Marton Elek. 2019-02-15 11:49:52 -08:00
Márton Elek
e0fe3d1eca HDDS-1110. OzoneManager need to login during init when security is enabled. Contributed by Xiaoyu Yao. 2019-02-15 16:12:24 +01:00
Anu Engineer
75e15cc0c4 HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects.
Contributed by Elek, Marton.
2019-02-14 23:33:25 -08:00
Anu Engineer
0395f22145 HDDS-1068. Improve the error propagation for ozone sh.
Contributed by Elek, Marton.
2019-02-14 22:51:46 -08:00
Anu Engineer
2d83b24994 HDDS-1108. Check s3bucket exists or not before MPU operations.
Contributed by Bharat Viswanadham.
2019-02-14 18:10:18 -08:00
Márton Elek
7a57974f1f HDDS-1096. OzoneManager#loadOMHAConfigs should use default ports in case port is not defined. Contributed by Hanisha Koneru. 2019-02-14 14:02:16 +01:00
Bharat Viswanadham
917ac9f108 HDDS-972. Add support for configuring multiple OMs. Contributed by Hanisha Koneru. 2019-02-12 22:07:27 -08:00
Ajay Kumar
d48e61dd36 HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar 2019-02-11 18:59:47 -08:00
Márton Elek
e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Márton Elek
9f2da01591 HDDS-776. Make OM initialization resilient to dns failures. Contributed by Doroszlai, Attila. 2019-02-05 16:11:19 +01:00
Márton Elek
033d97abac HDDS-956. MultipartUpload: List Parts for a Multipart upload key. Contributed by Bharat Viswanadham. 2019-01-31 14:51:31 +01:00
Bharat Viswanadham
945a61c164 HDDS-549. Add support for key rename in Ozone Shell. Contributed by Doroszlai Attila. 2019-01-30 16:00:18 -08:00
Xiaoyu Yao
2ec296e659 HDDS-991. Fix failures in TestSecureOzoneCluster. Contributed by Ajay Kumar. 2019-01-25 10:50:23 -08:00
Xiaoyu Yao
9fc7df8afb HDDS-793. Support custom key/value annotations on volume/bucket/key. Contributed by Elek, Marton. 2019-01-24 23:24:09 -08:00
Nanda kumar
45c4cfe790 HDDS-906. Display the ozone version on SCM/OM web ui instead of Hadoop version.
Contributed by Doroszlai, Attila.
2019-01-25 12:16:29 +05:30
Márton Elek
4e0aa2ceac HDDS-948. MultipartUpload: S3 API for Abort Multipart Upload. Contributed by Bharat Viswanadham. 2019-01-24 20:24:19 +01:00
Ajay Kumar
dcbc8b86ed HDDS-975. Manage ozone security tokens with ozone shell cli. Contributed by Ajay Kumar. 2019-01-23 15:34:20 -08:00
Xiaoyu Yao
2aaaf12f9e HDDS-943. Add block token validation in HddsDispatcher/XceiverServer. Contributed by Ajay Kumar. 2019-01-18 12:11:34 -08:00