Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
21,416 Commits 326 Branches 371 Tags
Commit Graph

21416 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xiaoyu Yao
5f8ded542a
HDDS-1346. Remove hard-coded version ozone-0.5.0 from ReadMe of ozonesecure-mr docker-compose. Contributed by Xiaoyu Yao. 
This closes #652.

(cherry picked from commit 8a59efee34107517bf6ad6dfb9b4342098e2f847)
 2019-03-27 21:53:31 -07:00
Shweta Yakkali
299177eaa7
HDDS-139. Output of createVolume can be improved. Contributed by Shweta. 
(cherry picked from commit f426b7ce8fb33d57e4187484448b9e0bfc04ccfa)
 2019-03-26 19:03:09 -07:00
Ajay Yadav
447d53476a
HDDS-939. Add S3 access check to Ozone manager. Contributed by Ajay Kumar. (#634) 
(cherry picked from commit 82d477293c879ed4529efe2b5e8d138e09bbce3c)
 2019-03-26 09:02:15 -07:00
Sandeep Nemuri
f6acbc9caf HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. 
Signed-off-by: Nanda kumar <nanda@apache.org>
(cherry picked from commit 5c0a81ad3c3d28cc1dd8d91594298d3fc7ebcfa4)
 2019-03-26 15:45:45 +05:30
Shashikant Banerjee
eed623ad61 HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee. 2019-03-25 15:45:26 +05:30
Xiaoyu Yao
8d1c218428
HDDS-1299. Support TokenIssuer interface for running jobs with OzoneFileSystem. Contributed by Xiaoyu Yao. 
This closes #627.

(cherry picked from commit 6a34c9bb29dc3e0370b63b63f26229e3dc90cc13)
 2019-03-23 09:36:56 -07:00
Vivek Ratnavel Subramanian
6b3e0b3edc
HDDS-1302. Fix SCM CLI does not list container with id 1. 
(cherry picked from commit 73f7b04e2b8f9a4b06a1e5b5c62eadd074555205)
 2019-03-22 15:17:12 -07:00
Xiaoyu Yao
9b989c49c9 HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading from stale pipeline. Contributed by Xiaoyu Yao. (#639) 
(cherry picked from commit dea6f2a065271d2e37eda05bbcb98fb1bb5ed2a6)
 2019-03-22 15:09:22 -07:00
Arpit Agarwal
6d73e7bdf4
HDDS-1323. Ignore unit test TestFailureHandlingByClient. Contributed by Supratim Deka. 
(cherry picked from commit 1d389ecb24482c2c4b41df898e8f9bc937cc524d)
 2019-03-22 11:30:50 -07:00
Arpit Agarwal
8f422d7b5e
HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh. 
(cherry picked from commit 90afc9ab0382d45083dca1434f02936985798e48)
 2019-03-21 14:27:07 -07:00
Arpit Agarwal
6c009a3d04
HDDS-1321. TestOzoneManagerHttpServer depends on hard-coded port numbers. Contributed by Arpit Agarwal. (#633) 
Change-Id: I9656af4a7f41812da9d125c10ae0e8daf3dcf7f5
(cherry picked from commit 2828f8c339d7a03bd2bedf99c7700a3bbeec3a34)
 2019-03-21 14:21:05 -07:00
Xiaoyu Yao
b2c63c3ce1
HDDS-1246. Add ozone delegation token utility subcmd for Ozone CLI. Contributed by Xiaoyu Yao. (#594) 
(cherry picked from commit 5cfb88a225157366e194fc7fb2e20141b1ad24db)
 2019-03-20 19:09:25 -07:00
Hanisha Koneru
598c16c1fc
HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 
(cherry picked from commit 12402b7a74a9069230ebfdc9e5c2ee0e28a23990)
 2019-03-20 17:53:06 -07:00
Bharat Viswanadham
3edb697a37
HDDS-1307. Test ScmChillMode testChillModeOperations failed. (#622) 
HDDS-1307. Test ScmChillMode testChillModeOperations failed. Contributed by Bharat Viswanadham. (#622)

(cherry picked from commit 1639071b054d120d8b99f34b4deed837d3afa11f)
 2019-03-19 13:58:00 -07:00
Xiaoyu Yao
a55a295c7a HDDS-1215. Change hadoop-runner and apache/hadoop base image to use Java8. Contributed by Xiaoyu Yao. 
This closes #615

(cherry picked from commit 992489c11ca6710466266250a3933e0281ac4d35)
 2019-03-19 09:57:50 -07:00
Ajay Yadav
d2e115d8c8 HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 
(cherry picked from commit f10d49332522beca7cb7342e68b2acdbe4c974f8)
 2019-03-18 23:11:36 -07:00
Arpit Agarwal
7cd82b4547
Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)" 
This reverts commit 0d35cfc0065a021318c604dc01dd2794acb9105b.
 2019-03-18 14:23:56 -07:00
Xiaoyu Yao
6fd95b8595 HDDS-1296. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#616) 
(cherry picked from commit 66a104bc57242b2817e8c1f67cb6776e70cf5c47)
 2019-03-18 09:39:05 -07:00
Vivek Ratnavel Subramanian
328bfcc0b6
HDDS-1263. SCM CLI does not list container with id 1. 
(cherry picked from commit af2dfc9f3d3661d1837a8b749882d557834c39fe)
 2019-03-15 20:56:09 -07:00
Elek, Márton
0d35cfc006 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608) 
(cherry picked from commit 44b8451821c392dd59ee84153c98547ae9ce7042)
 2019-03-15 14:53:38 -07:00
Elek, Márton
a009255723 HDDS-1283. Fix the dynamic documentation of basic s3 client usage. Contributed by Elek, Marton. (#605) 
(cherry picked from commit 16b78622ccf641e3805a0d78be9c1c3e20f97f6a)
 2019-03-15 14:13:08 -07:00
Xiaoyu Yao
3ca1024eec HDDS-1138. Ozone Client should avoid talking to SCM directly. Contributed by Xiaoyu Yao and Mukul Kumar Singh. 
Closes #585 

(cherry picked from commit d1afa038044f6aaec43551571caae567b2bf4139)
 2019-03-15 10:42:09 -07:00
Elek, Márton
522976e405 HDDS-1259. OzoneFS classpath separation is broken by the token validation. Contributed by Elek Marton. 
Closes #604 

(cherry picked from commit dc21655f2a477196ccc5173666b73d11865eeaf5)
 2019-03-15 10:05:11 -07:00
Márton Elek
3fd9c0db17 HDDS-807. Period should be an invalid character in bucket names. Contributed by Siddharth Wagle. 
(cherry picked from commit c449cdebe68fb5fe2a7bd83715f6c3f4d5b523d9)
 2019-03-15 10:04:50 -07:00
Shashikant Banerjee
8bd13ffa10 HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee. 
(cherry picked from commit 155ab6d5d8ab4a80019e65351572320502d8a510)
 2019-03-15 15:47:56 +05:30
Vivek Ratnavel Subramanian
64726dd8fe
HDDS-761. Create S3 subcommand to run S3 related operations. 
Closes #579

(cherry picked from commit 90015084850298803d8e165a0f6f9719ba724ea3)
 2019-03-14 21:19:48 -07:00
Vivek Ratnavel Subramanian
c00e704496
HDDS-1265. ozone sh s3 getsecret throws Null Pointer Exception for unsecured clusters. 
Closes #611

(cherry picked from commit 2627dad333eb88011b854fe163d70e79163e33f8)
 2019-03-14 17:22:21 -07:00
Shashikant Banerjee
8614da3723 HDDS-1257. Incorrect object because of mismatch in block lengths. Contributed by Shashikant Banerjee. 
(cherry picked from commit d60673c47077d69320ae1bd37c6b74489bef25f7)
 2019-03-14 19:34:54 +05:30
Mukul Kumar Singh
8e66b6220b HDDS-1237. Fix test TestSecureContainerServer.testClientServerRatisGrpc. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 983b78afdc8939d0d2116f23837d958e6b5d9e46)
 2019-03-14 17:03:07 +05:30
Mukul Kumar Singh
f38a8a9630 HDDS-1241. Update ozone to latest ratis snapshot build (0.4.0-5680cf5-SNAPSHOT). Contributed by Mukul Kumar Singh. 
(cherry picked from commit ea9fb07fd22417ab22d231531277d9ffe07814a2)
 2019-03-14 15:25:22 +05:30
Ajay Yadav
0f4ae392d7 HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598) 
(cherry picked from commit 4fa009989bd641388aeb9b1bd9c90e9ddbfe84c0)
 2019-03-13 12:02:17 -07:00
Xiaoyu Yao
eeb7d63046 HDDS-1087. Fix TestDefaultCertificateClient#testSignDataStream. Contributed by Xiaoyu Yao. (#596) 
(cherry picked from commit 86d508c7c724bd31a81066126ffc675d36d9df0a)
 2019-03-13 11:52:47 -07:00
Aravindan Vijayan
34f0ad064d HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline. 
Signed-off-by: Nanda kumar <nanda@apache.org>
(cherry picked from commit 1f47fb7a2fa9bfb9fd2a7ca5c9f1128fabc32784)
 2019-03-13 17:14:51 +05:30
Lokesh Jain
b5b3f4d823 HDDS-1128. Create stateful manager class for the pipeline creation scheduling. 
Signed-off-by: Nanda kumar <nanda@apache.org>
(cherry picked from commit 0d62753da9650c2d3470ff4572734241ecaa6a71)
 2019-03-13 16:47:04 +05:30
Vivek Ratnavel Subramanian
32477ed1e6
HDDS-1226. Addendum. ozone-filesystem jar missing in hadoop classpath 
Closes #560

(cherry picked from commit c1649bf1e192840f00b93df2cb962a5b2dab4d92)
 2019-03-13 12:09:41 +01:00
Ajay Yadav
15b6e38ddf
HDDS-1043. Enable token based authentication for S3 api 
Closes #561

(cherry picked from commit dcb0de848d8388fcee425847e52f6361790225c6)
 2019-03-12 18:54:59 +01:00
Xiaoyu Yao
262260fadf HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593) 
Closes PR #593

(cherry picked from commit 2778259b88246359bcdb03496568e0a7f45da391)
 2019-03-12 09:36:37 -07:00
Mukul Kumar Singh
97fd3025a4 HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh. 
(cherry picked from commit daf4660eebfa9267e7082f7c9a14c3dd5d6d5d46)
 2019-03-12 20:03:36 +05:30
Shashikant Banerjee
4373c12701 HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee. 
(cherry picked from commit b4aa24d3c5ad1b9309a58795e4b48e567695c4e4)
 2019-03-12 10:21:14 +05:30
Xiaoyu Yao
e673bebcfe HDDS-1245. OM delegation expiration time should use Time.now instead … (#587) 
Closes #587.

(cherry picked from commit c7307867f0d28dbb11e2194ff885da8269e4fb42)
 2019-03-11 21:20:57 -07:00
Ajay Yadav
6028aefb57 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar. 
This closes (#581)

(cherry picked from commit 599e926d462134aa3235311feb27f3a8cdff6a51)
 2019-03-11 15:02:43 -07:00
Xiaoyu Yao
1050bf42fa HDDS-1236. Fix incorrect Ozone ClientProtocol KerberosInfo annotation. Contributed by Xiaoyu Yao. (#573) 
(cherry picked from commit 4fc493e511dacc41628925bdcb3506cd944346e6)
 2019-03-11 11:51:38 -07:00
Xiaoyu Yao
fe41e67ba4 Revert "HDDS-1236. Fix incorrect Ozone ClientProtocol KerberosInfo annotation. Contributed by Xiaoyu Yao. (#573)" 
This reverts commit 89fb60ffd07e0ea21c6fb431eacc9a575360fb93.
 2019-03-11 11:50:34 -07:00
Xiaoyu Yao
89fb60ffd0 HDDS-1236. Fix incorrect Ozone ClientProtocol KerberosInfo annotation. Contributed by Xiaoyu Yao. (#573) 
(cherry picked from commit 4fc493e511dacc41628925bdcb3506cd944346e6)
 2019-03-11 09:22:29 -07:00
Mukul Kumar Singh
5d67a17877 HDDS-1238. Fix Ratis Snapshot creation error if the snapshot file already exists in ContainerStateMachine. Contributed by Shashikant Banerjee. 
(cherry picked from commit 8458ced11e5b6690a41593c247c1190bf3f31c0b)
 2019-03-11 14:57:45 +05:30
Mukul Kumar Singh
aa9de9a0fa HDDS-1210. Ratis pipeline creation doesn't check raft client reply status during initialization. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 2c3ec37738544107238f75d0ca781fd23bdc309b)
 2019-03-11 14:57:45 +05:30
Lokesh Jain
60b5e178e3 HDDS-1220. KeyManager#openKey should release the bucket lock before doing an allocateBlock. Contributed by Lokesh Jain. 
(cherry picked from commit ebb5fa115b842a3bcb747e881ca2a8fe4c716d07)
 2019-03-11 14:56:51 +05:30
Bharat Viswanadham
e65717af25
HDDS-1242. In S3 when bucket already exists, it should just return location. 
(cherry picked from commit 73e1801b2db928d165e4cf076e1ba50ec1e8b48a)
 2019-03-09 21:48:46 -08:00
Xiaoyu Yao
49217971b5 HDDS-1235. BaseHttpServer NPE is HTTP policy is HTTPS_ONLY. Contributed by Xiaoyu Yao. 
Closes #572

(cherry picked from commit 11db46956c6062a986caf9009d3e36049e5830d4)
 2019-03-09 09:40:05 -08:00
Ajay Yadav
6465125a5a HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547) 
(cherry picked from commit 064f38b3a51be5f99a0ba09d0d48c11286017017)
 2019-03-09 09:40:00 -08:00