Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,392 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

454 Commits

Author SHA1 Message Date
Shashikant Banerjee 8c8cb2d6aa HDDS-1502. Fix the compilation issue with commit 18c1eebc08. 2019-05-30 18:44:22 +05:30
Shashikant Banerjee 18c1eebc08 HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833). 2019-05-30 16:17:45 +05:30
Ajay Yadav 12be6ff2ff HDDS-1539. Implement addAcl,removeAcl,setAcl,getAcl for Volume. Contributed Ajay Kumar. (#847) 2019-05-29 23:18:06 -07:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
Xiaoyu Yao 7f2e87a419
HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866) 2019-05-28 18:39:23 -07:00
Elek, Márton 79d14d0d42 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862) 2019-05-28 14:40:45 -07:00
Xiaoyu Yao fb0b39f4bf
HDDS-1536. testSCMSafeModeRestrictedOp is failing consistently. Contributed by Xiaoyu Yao. (#865) 2019-05-28 14:02:36 -07:00
Shashikant Banerjee 83549dbbea HDDS-1509. TestBlockOutputStreamWithFailures#test2DatanodesFailure fails intermittently. Contributed by Shashikant Banerjee (#805). 2019-05-27 16:31:44 +05:30
Shashikant Banerjee f0e44b3a3f HDDS-1584. Fix TestFailureHandlingByClient tests. Contributed by Shashikant Banerjee (#845). 2019-05-27 16:26:21 +05:30
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
Ajay Yadav 9c61494c02 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834) 2019-05-22 11:47:32 -07:00
Mukul Kumar Singh 2fc6f8599a HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825) 2019-05-22 17:18:40 +05:30
Bharat Viswanadham 77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Siddharth c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle. 
This closes #822.
 2019-05-16 15:13:10 -07:00
supratimdeka 9569015802 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812) 2019-05-15 10:26:01 -07:00
Hanisha Koneru 02c9efcb81
HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795) 2019-05-13 20:49:52 -07:00
avijayanhwx eb9c8900bc HDDS-1475 : Fix OzoneContainer start method. (#788) 2019-05-07 14:09:41 -07:00
Shashikant Banerjee 7f0e2c67e0 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee 2019-05-08 01:00:10 +05:30
Shashikant Banerjee 01451a57de Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)" 
This reverts commit d525633619.
 2019-05-08 00:49:15 +05:30
bshashikant d525633619
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749) 
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749).
 2019-05-08 00:38:53 +05:30
Márton Elek fb7c1cad0e
Revert "HDDS-1384. TestBlockOutputStreamWithFailures is failing" 
This reverts commit dead9b4049.
 2019-05-06 20:17:00 +02:00
avijayanhwx f194540520 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786) 2019-05-04 00:19:00 +05:30
Siddharth 1df679985b HDDS-1473. DataNode ID file should be human readable. (#781) 2019-05-02 13:59:14 -07:00
Márton Elek e2f0f72677
HDDS-1469. Generate default configuration fragments based on annotations 
Closes #773
 2019-05-02 12:14:43 +02:00
Márton Elek dead9b4049
HDDS-1384. TestBlockOutputStreamWithFailures is failing 
Closes #750
 2019-04-30 14:27:39 +02:00
Bharat Viswanadham 43b2a4b77b
HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769) 2019-04-26 14:25:34 -07:00
Kitti Nanasi 3f787cd506
HDDS-1192. Support -conf command line argument in GenericCli 
Closes #713
 2019-04-24 15:34:49 +02:00
Siddharth 59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Márton Elek b979fdde99
HDDS-1447. Fix CheckStyle warnings. Contributed by Wanqiang Ji. 2019-04-18 08:54:43 +02:00
Shashikant Banerjee d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee 082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)" 
This reverts commit df2ae27f3e.
 2019-04-17 14:38:08 +05:30
Shashikant Banerjee df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373" 
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
 2019-04-17 14:17:54 +05:30
bshashikant 4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373 
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
 2019-04-17 14:12:52 +05:30
Hanisha Koneru e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Arpit Agarwal f2ab2795db
HDDS-1432. Ozone client list command truncates response without any indication. Contributed by Siddharth Wagle. 2019-04-16 12:35:49 -07:00
Shashikant Banerjee bfcb6534cd HDDS-1380. Add functonality to write from multiple clients in MiniOzoneChaosCluster. Contributed by Shashikant Banerjee. 2019-04-16 18:52:07 +05:30
Arpit Agarwal ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Elek, Márton 5379d85d8e HDDS-1387. ConcurrentModificationException in TestMiniChaosOzoneCluster. Contributed by Marton Elek. (#732) 2019-04-12 15:19:50 -07:00
Bharat Viswanadham a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher 
Closes #723
 2019-04-12 12:21:10 +02:00
Doroszlai, Attila 75ba719ddd
HDDS-1419. Fix shellcheck errors in start-chaos.sh 
Closes #728
 2019-04-12 11:59:55 +02:00
Arpit Agarwal fd676e190a
HDDS-1426. Minor logging improvements for MiniOzoneChaosCluster 
Closes #730
 2019-04-12 11:42:23 +02:00
Nanda kumar 4003849fa4
HDDS-1416. MiniOzoneCluster should set custom value for hdds.datanode.replication.work.dir. 
Contributed by chencan.
 2019-04-12 14:45:57 +05:30
Arpit Agarwal fcc7f9b32f
HDDS-1410. TestSCMNodeMetrics is flaky. Contributed by Siddharth Wagle. 2019-04-11 15:14:28 -07:00
Shashikant Banerjee 22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Arpit Agarwal feaab241e5
HDDS-1418. Move bang line to the start of the start-chaos.sh script. Contributed by Arpit Agarwal. (#720) 
Change-Id: I4fcf39d61a7d4c4ca79cb56a6958db0f691fe971
 2019-04-10 12:44:01 -07:00
Mukul Kumar Singh df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Bharat Viswanadham 25c421bcc0
HDDS-1393. Convert all OM Bucket related operations to HA model. (#704) 2019-04-09 14:57:32 -07:00
Mukul Kumar Singh 312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Bharat Viswanadham 77fe51e136
HDDS-1379. Convert all OM Volume related operations to HA model. (#689) 2019-04-04 21:09:50 -07:00
avijayanhwx 993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Hanisha Koneru f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Hanisha Koneru 4e64f8d763 HDDS-1324. TestOzoneManagerHA tests are flaky (#676) 2019-04-03 16:02:19 -07:00
Bharat Viswanadham 358f7f9b99
HDDS-1211. Test SCMChillMode failing randomly in Jenkins run (#543) 2019-04-03 15:02:00 -07:00
Ajay Kumar bfc90bdc64
HDDS-1355. Only FQDN is accepted for OM rpc address in secure environment. Contributed by Ajay Kumar. 
Closes #677
 2019-04-02 15:43:01 +02:00
Yiqun Lin d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Shashikant Banerjee ef5de29243 HDDS-1312. Add more unit tests to verify BlockOutputStream functionalities. Contributed by Shashikant Banerjee. 2019-04-01 16:09:58 +05:30
Shashikant Banerjee 509f31b109 HDDS-1295. Add MiniOzoneChaosCluster to mimic long running workload in a unit test environment. Contributed by Mukul Kumar Singh. 2019-04-01 15:50:03 +05:30
Bharat Viswanadham bf3b7fd732
HDDS-1288. SCM - Failing test on trunk that waits for HB report processing. Contributed by Siddharth Wagle. 2019-03-30 10:54:45 -07:00
Elek, Márton ec82e4cab2 HDDS-1357. ozone s3 shell command has confusing subcommands (#663) 2019-03-30 10:13:29 -07:00
Xiaoyu Yao ca5e4ce036
HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao. 
This closes #641
 2019-03-28 12:00:58 -07:00
Vivek Ratnavel Subramanian b2269581f7 HDDS-1264. Remove Parametrized in TestOzoneShell. (#614) 2019-03-26 22:07:22 -07:00
Bharat Viswanadham eef8cae7cf
HDDS-1262. In OM HA OpenKey call Should happen only leader OM. (#626) 2019-03-26 21:48:01 -07:00
Ajay Yadav 82d477293c HDDS-939. Add S3 access check to Ozone manager. Contributed by Ajay Kumar. (#634) 2019-03-26 08:59:59 -07:00
Shashikant Banerjee d4e4a7d456 HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee. 2019-03-25 15:41:20 +05:30
Xiaoyu Yao dea6f2a065 HDDS-1291. Set OmKeyArgs#refreshPipeline flag properly to avoid reading from stale pipeline. Contributed by Xiaoyu Yao. (#639) 2019-03-22 15:08:28 -07:00
Arpit Agarwal 1d389ecb24
HDDS-1323. Ignore unit test TestFailureHandlingByClient. Contributed by Supratim Deka. 2019-03-22 11:29:56 -07:00
Bharat Viswanadham 1639071b05 HDDS-1307. Test ScmChillMode testChillModeOperations failed. (#622) 
HDDS-1307. Test ScmChillMode testChillModeOperations failed. Contributed by Bharat Viswanadham. (#622)
 2019-03-19 13:57:02 -07:00
Ajay Yadav f10d493325
HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 2019-03-18 23:08:17 -07:00
Bharat Viswanadham 93db5da4d9
HDDS-1250. In OM HA AllocateBlock call where connecting to SCM from OM should not happen on Ratis. 2019-03-18 13:51:44 -07:00
Shashikant Banerjee 155ab6d5d8 HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee. 2019-03-15 15:40:59 +05:30
Vivek Ratnavel Subramanian 9001508485 HDDS-761. Create S3 subcommand to run S3 related operations. 
Closes #579
 2019-03-14 21:16:59 -07:00
Vivek Ratnavel Subramanian 2627dad333 HDDS-1265. ozone sh s3 getsecret throws Null Pointer Exception for unsecured clusters. 
Closes #611
 2019-03-14 17:21:06 -07:00
Bharat Viswanadham 091a664977
HDDS-917. Expose NodeManagerMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-14 12:41:52 -07:00
Mukul Kumar Singh 983b78afdc HDDS-1237. Fix test TestSecureContainerServer.testClientServerRatisGrpc. Contributed by Mukul Kumar Singh. 2019-03-14 16:49:43 +05:30
Márton Elek 3d140a537a
HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton. 
Closes #588
 2019-03-14 12:01:25 +01:00
Aravindan Vijayan 1f47fb7a2f HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 17:12:43 +05:30
Lokesh Jain 0d62753da9 HDDS-1128. Create stateful manager class for the pipeline creation scheduling. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 16:30:06 +05:30
Ajay Yadav dcb0de848d
HDDS-1043. Enable token based authentication for S3 api 
Closes #561
 2019-03-12 18:47:06 +01:00
Shashikant Banerjee b4aa24d3c5 HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-03-11 23:15:49 +05:30
Bharat Viswanadham 68aa2ccd89
HDDS-1240. Fix check style issues caused by HDDS-1196. 2019-03-09 21:50:46 -08:00
Bharat Viswanadham 341c076f05 HDDS-1196. Add a ReplicationStartTimer class. (#567) 2019-03-08 20:55:33 +05:30
Yiqun Lin fb851c9481 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan. 2019-03-08 16:59:41 +08:00
Hanisha Koneru bb12e81ec8
HDDS-1175. Serve read requests directly from RocksDB. (#557) 
HDDS-1175. Serve read requests directly from RocksDB.
 2019-03-06 19:44:55 -08:00
Márton Elek 45f976fdcf
HDDS-1113. Remove default dependencies from hadoop-ozone project. Contributed by Elek, Marton. 
Closes #490
 2019-03-06 18:22:22 +01:00
Mukul Kumar Singh 2c3ec37738 HDDS-1210. Ratis pipeline creation doesn't check raft client reply status during initialization. Contributed by Mukul Kumar Singh. 2019-03-06 21:38:21 +05:30
Elek, Márton 7f636b47b5 HDDS-919. Enable prometheus endpoints for Ozone datanodes (#502) 
Pull request from Elek, Márton. (#502)
 2019-03-05 12:04:57 -08:00
Hanisha Koneru 12402b7a74 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-05 09:24:22 -08:00
Shashikant Banerjee 5fcea54a50 HDDS-935. Avoid creating an already created container on a datanode in case of disk removal followed by datanode restart. Contributed by Shashikant Banerjee. 2019-03-05 22:09:46 +05:30
Ajay Yadav 353b90d59e HDDS-1156. testDelegationToken is failing in TestSecureOzoneCluster. Contributed by Ajay Kumar. 2019-03-05 08:37:10 -08:00
Anu Engineer 10b802b84b HDDS-1136 : Add metric counters to capture the RocksDB checkpointing statistics. 
Contributed by Aravindan Vijayan.
 2019-03-04 12:00:16 -08:00
Hanisha Koneru b18c1c22ea
Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client." 
This reverts commit 8e1225991d.
 2019-03-04 16:58:59 +09:00
Ajay Kumar 4a3cddea70 HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. 2019-03-01 15:41:09 -08:00
Shashikant Banerjee de1dae64f2 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee. 2019-03-01 23:30:28 +05:30
Hanisha Koneru 8e1225991d HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-01 00:27:39 -08:00
Bharat Viswanadham 6c8c422beb
HDDS-1178. Healthy pipeline Chill Mode Rule. 
Closes #518
 2019-02-27 14:26:40 +01:00
Xiaoyu Yao 9e0f3d1c52 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar. 2019-02-26 06:02:10 -08:00