Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,628 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

831 Commits

Author SHA1 Message Date
sdeka 456bb8a3e0 verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion 2019-05-22 09:34:21 +05:30
Bharat Viswanadham 77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Lokesh Jain c1d7d68c78
HDDS-1461. Optimize listStatus api in OzoneFileSystem (#782) 2019-05-21 14:47:48 +05:30
avijayanhwx f63300228e HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799) 
* HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected.

* HDDS-1451 : Address review comments.
 2019-05-20 22:02:56 +02:00
sdeka 64c39856d2 Fixed checkstyle issues. 2019-05-20 08:13:37 +05:30
Bharat Viswanadham 0d1d7c86ec HDDS-1499. OzoneManager Cache. (#798) 2019-05-19 19:23:02 -07:00
sdeka 869a1ab41a HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka 2019-05-18 16:16:33 +05:30
Arpit Agarwal 732133cb2a HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725) 
* HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit Agarwal.

Change-Id: I6db6bdd19839a45e5341ed7e745cd38f68af8378

* Suppress spurious findbugs warning.

* Remove log file that got committed in error
 2019-05-18 11:47:19 +02:00
Siddharth c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle. 
This closes #822.
 2019-05-16 15:13:10 -07:00
Márton Elek 03ea8ea92e
HDDS-1297. Fix IllegalArgumentException thrown with MiniOzoneCluster Initialization. Contributed by Yiqun Lin. 2019-05-16 17:04:46 +02:00
Márton Elek 9248b7d58b
HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. 2019-05-16 17:04:46 +02:00
Shashikant Banerjee e66ecc564a HDDS-1531. Disable the sync flag by default during chunk writes in Datanode (#820). Contributed by Shashikant Banerjee. 2019-05-16 18:38:52 +05:30
supratimdeka 9569015802 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812) 2019-05-15 10:26:01 -07:00
Hanisha Koneru 02c9efcb81
HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795) 2019-05-13 20:49:52 -07:00
Vivek Ratnavel Subramanian b832e174b0 HDDS-1474. "ozone.scm.datanode.id" config should take path for a dir and not a file (#809) 2019-05-09 17:11:09 -07:00
Hanisha Koneru 2d31ccc1b3
Revert "HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792)" (#808) 
This reverts commit 18cca583ac.
 2019-05-09 11:18:46 -07:00
Vivek Ratnavel Subramanian 18cca583ac HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792) 2019-05-09 10:23:44 -07:00
avijayanhwx eb9c8900bc HDDS-1475 : Fix OzoneContainer start method. (#788) 2019-05-07 14:09:41 -07:00
Shashikant Banerjee 7f0e2c67e0 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee 2019-05-08 01:00:10 +05:30
Shashikant Banerjee 01451a57de Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)" 
This reverts commit d525633619.
 2019-05-08 00:49:15 +05:30
bshashikant d525633619
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749) 
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749).
 2019-05-08 00:38:53 +05:30
avijayanhwx f194540520 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786) 2019-05-04 00:19:00 +05:30
Siddharth 1df679985b HDDS-1473. DataNode ID file should be human readable. (#781) 2019-05-02 13:59:14 -07:00
Márton Elek e2f0f72677
HDDS-1469. Generate default configuration fragments based on annotations 
Closes #773
 2019-05-02 12:14:43 +02:00
Vivek Ratnavel Subramanian 3cb1d09b2e
HDDS-1479. Update S3.md documentation 
Closes #791
 2019-05-02 11:49:37 +02:00
Márton Elek a2887f5c23
HDDS-1468. Inject configuration values to Java objects 
Closes #772
 2019-05-02 11:35:01 +02:00
Ajay Kumar 382a962d8d
Revert "HDDS-973. HDDS/Ozone fail to build on Windows." 
This reverts commit 5dae1a0c66.
 2019-04-29 19:46:20 -07:00
Elek, Márton 59816dff94 HDDS-1462. Fix content and format of Ozone documentation. Contributed by Elek, Marton. (#767) 2019-04-29 13:28:19 -07:00
Siddharth 5bca062d0e HDDS-1476. Fix logIfNeeded logic in EndPointStateMachine. (#779) 2019-04-29 12:05:38 -07:00
Xiaoyu Yao a96d16a843
HDDS-1455. Inconsistent naming convention with Ozone Kerberos configuration. Contributed by Xiaoyu Yao. (#762) 2019-04-29 11:18:11 -07:00
Ajay Yadav b434f558ca
HDDS-1471. Update ratis dependency to 0.3.0. Contributed by Ajay Kumar. (#777) 2019-04-28 20:57:59 -07:00
Bharat Viswanadham 43b2a4b77b
HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769) 2019-04-26 14:25:34 -07:00
Hanisha Koneru 37582705fa
HDDS-1403. KeyOutputStream writes fails after max retries while writing to a closed container (#753) 2019-04-26 10:39:03 -07:00
Eric Yang e95c055d91
HDDS-1457. Sequence of configuring Ozone File System instruction needs some work. Contributed by Eric Yang. 2019-04-24 15:52:51 +02:00
Kitti Nanasi 3f787cd506
HDDS-1192. Support -conf command line argument in GenericCli 
Closes #713
 2019-04-24 15:34:49 +02:00
Xiaoyu Yao 64f30da428 HDDS-1450. Fix nightly run failures after HDDS-976. Contributed by Xiaoyu Yao. (#757) 2019-04-23 21:38:23 -07:00
Siddharth 59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Nanda kumar 7e1f8d3a1b
HDDS-1368. Cleanup old ReplicationManager code from SCM. 2019-04-23 17:35:39 +05:30
Chen, Junjie 518f47bf9b HDDS-976: Parse network topology from yaml file. Contributed by Junjie Chen. (#661) 
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
 2019-04-18 16:53:25 -07:00
Márton Elek 8aea5e4cb6
HDDS-1433. Rename GetScmInfoRespsonseProto to GetScmInfoResponseProto due to typos. Contributed by Wanqiang Ji. 2019-04-18 08:53:08 +02:00
Shashikant Banerjee d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee 082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)" 
This reverts commit df2ae27f3e.
 2019-04-17 14:38:08 +05:30
Shashikant Banerjee df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373" 
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
 2019-04-17 14:17:54 +05:30
bshashikant 4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373 
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
 2019-04-17 14:12:52 +05:30
Doroszlai, Attila 0a87fd1464 HDDS-1434. TestDatanodeStateMachine is flaky (#740) 2019-04-16 19:13:01 -07:00
Hanisha Koneru e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Bharat Viswanadham 3ac3e50b20
HDDS-1374. ContainerStateMap cannot find container while allocating blocks. (#735) 2019-04-16 13:49:29 -07:00
Arpit Agarwal ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Arpit Agarwal 62f4808617
HDDS-1428. Remove benign warning in handleCreateContainer. Contributed by Siddharth Wagle. 2019-04-12 15:08:01 -07:00
Bharat Viswanadham a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher 
Closes #723
 2019-04-12 12:21:10 +02:00
Arpit Agarwal a9b46c58fe
HDDS-1421. Avoid unnecessary object allocations in TracingUtil 
Closes #722
 2019-04-12 11:52:31 +02:00
Doroszlai, Attila 3b08ac4659 HDDS-1363. ozone.metadata.dirs doesn't pick multiple dirs (#691) 2019-04-12 15:00:08 +05:30
Nanda kumar c298415eb5
HDDS-1404. Fix typos in HDDS. Contributed by bianqi. 2019-04-12 14:12:10 +05:30
Shashikant Banerjee 22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Nanda kumar e9c4109004 HDDS-1417. After successfully importing a container, datanode should delete the container tar.gz file from working directory. (#721) 2019-04-10 16:55:37 -07:00
Bharat Viswanadham 0e770a6539
HDDS-1370. Command Execution in Datanode fails because of NPE (#715) 2019-04-10 10:25:28 -07:00
Márton Elek dfb518bbf5
HDDS-1397. Avoid the usage of signal handlers in datanodes of the MiniOzoneClusters. 
Closes #701
 2019-04-10 18:35:36 +02:00
Mukul Kumar Singh df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Mukul Kumar Singh 312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Nanda kumar 73f43ac2dc
HDDS-1372. getContainerWithPipeline for a standalone pipeline fails with ConcurrentModificationException. (#682) 2019-04-09 15:10:07 +05:30
Shashikant Banerjee 2d4f6b6daa HDDS-372. There are three buffer copies in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-04-09 10:27:58 +05:30
Anu Engineer dc013f7a35 HDDS-1329. Update documentation for Ozone-0.4.0 release. 
Contributed By Ajay Kumar.
 2019-04-05 14:14:42 -07:00
Doroszlai, Attila 274ab4e86c HDDS-1332. Attempt to fix flaky test testStartStopDatanodeStateMachine (#697) 2019-04-05 13:21:58 -07:00
Nanda kumar 5750bb94ed HDDS-1389. Fix testSCMChillModeRestrictedOp. (#696) 2019-04-05 10:42:00 -07:00
Ajay Kumar 0356b3699d HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653) 
(cherry picked from commit 7ec6a31eb3)
 2019-04-05 09:29:21 -07:00
avijayanhwx 993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Shashikant Banerjee 33e159bf8d HDDS-1349. Remove watchClient from XceiverClientRatis. Contributed by Shashikant Banerjee. 2019-04-04 16:35:37 +05:30
Nanda kumar 48a58bce37
HDDS-1207. Refactor Container Report Processing logic and plugin new Replication Manager. (#662) 2019-04-04 16:32:59 +05:30
Yiqun Lin a92806d05a HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle. 2019-04-04 17:33:37 +08:00
Hanisha Koneru f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Yiqun Lin f96fb05a2b HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka. 2019-04-03 14:01:30 +08:00
Kitti Nanasi aeb43dfba3
HDDS-1153. Make tracing instrumentation configurable 
Closes #673
 2019-04-02 14:21:47 +02:00
Yiqun Lin d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Doroszlai, Attila 36756703f0 HDDS-1322. Hugo errors when building Ozone (#671) 2019-04-01 08:52:05 -07:00
Yiqun Lin 8bfef21efa HDDS-1360. Invalid metric type due to fully qualified class name. Contributed by Doroszlai, Attila. 2019-04-01 19:26:44 +08:00
Shweta Yakkali 82769ba1b8 HDDS-1352. Remove unused call in TestStorageContainerManagerHttpServer. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-29 17:26:48 +05:30
Xiaoyu Yao d7a2f9456b HDDS-1350. Fix checkstyle issue in TestDatanodeStateMachine. Contributed by Xiaoyu Yao. (#656) 2019-03-28 14:50:34 -07:00
nilotpalnandi 4cceeb2682 HDDS-1309 . change logging from warn to debug in XceiverClient. Contributed by Nilotpal Nandi. (#638) 2019-03-28 12:22:44 -07:00
Shashikant Banerjee ac4010bb22 HDDS-1293. ExcludeList#getProtoBuf throws ArrayIndexOutOfBoundsException. Contributed by Shashikant Banerjee. 2019-03-29 00:43:28 +05:30
Xiaoyu Yao ca5e4ce036
HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao. 
This closes #641
 2019-03-28 12:00:58 -07:00
Arpit Agarwal b4ed81c4e6 HDDS-1332. Add some logging for flaky test testStartStopDatanodeStateMachine. Contributed by Arpit Agarwal. (#649) 2019-03-27 20:53:30 +05:30
Sandeep Nemuri 5c0a81ad3c HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-26 15:44:18 +05:30
Bharat Viswanadham 8739693514 HDDS-1217. Refactor ChillMode rules and chillmode manager. (#558) 2019-03-26 01:41:03 +05:30
Yiqun Lin e5d72f504e HDDS-1234. Iterate the OM DB snapshot and populate the recon container DB. Contributed by Aravindan Vijayan. 2019-03-25 22:52:02 +08:00
Shashikant Banerjee d4e4a7d456 HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee. 2019-03-25 15:41:20 +05:30
Bharat Viswanadham 128dd91e10
HDDS-1331. In DatanodeStateMachine join check for not null. 2019-03-23 09:43:49 -07:00
Bharat Viswanadham 43e421afef
HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-22 16:44:12 -07:00
Ajay Kumar 771ea6b5e7 Revert "HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri." 
This reverts commit efad5717ec.
 2019-03-22 16:20:44 -07:00
Ajay Kumar efad5717ec HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri. 2019-03-22 15:53:12 -07:00
Vivek Ratnavel Subramanian 73f7b04e2b HDDS-1302. Fix SCM CLI does not list container with id 1. 2019-03-22 15:15:01 -07:00
Arpit Agarwal f854a89190
HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED containers. Contributed by Nanda kumar. (#620) 2019-03-22 14:36:29 -07:00
Arpit Agarwal 90afc9ab03
HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-21 14:24:02 -07:00
Yiqun Lin 60cdd4cac1 HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan. 2019-03-21 11:02:29 +08:00
Mukul Kumar Singh 072750c00a HDDS-1306. TestContainerStateManagerIntegration fails in Ratis shutdown. Contributed by Lokesh Jain. 2019-03-20 17:14:19 +05:30
Ajay Yadav f10d493325
HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 2019-03-18 23:08:17 -07:00
Arpit Agarwal ae3a2c3851
Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)" 
This reverts commit 44b8451821.
 2019-03-18 14:21:57 -07:00
Sammi Chen 4d2a116d6e HDDS-699. Detect Ozone Network topology. Contributed by Sammi Chen. 2019-03-17 17:08:29 +08:00
Bharat Viswanadham 926d548caa
HDDS-1281. Fix the findbug issue caused by HDDS-1163. Contributed by Aravindan Vijayan. 2019-03-15 21:06:44 -07:00
Vivek Ratnavel Subramanian af2dfc9f3d HDDS-1263. SCM CLI does not list container with id 1. 2019-03-15 15:39:30 -07:00
Elek, Márton 44b8451821 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608) 2019-03-15 14:51:37 -07:00
Shashikant Banerjee 155ab6d5d8 HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee. 2019-03-15 15:40:59 +05:30
Vivek Ratnavel Subramanian 9001508485 HDDS-761. Create S3 subcommand to run S3 related operations. 
Closes #579
 2019-03-14 21:16:59 -07:00
Bharat Viswanadham 091a664977
HDDS-917. Expose NodeManagerMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-14 12:41:52 -07:00
Shashikant Banerjee d60673c470 HDDS-1257. Incorrect object because of mismatch in block lengths. Contributed by Shashikant Banerjee. 2019-03-14 19:32:36 +05:30
Márton Elek 3d140a537a
HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton. 
Closes #588
 2019-03-14 12:01:25 +01:00
Mukul Kumar Singh ea9fb07fd2 HDDS-1241. Update ozone to latest ratis snapshot build (0.4.0-5680cf5-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-14 15:18:06 +05:30
Ajay Yadav 4fa009989b
HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598) 2019-03-13 12:01:08 -07:00
Xiaoyu Yao 86d508c7c7 HDDS-1087. Fix TestDefaultCertificateClient#testSignDataStream. Contributed by Xiaoyu Yao. (#596) 2019-03-13 11:50:53 -07:00
Aravindan Vijayan 1f47fb7a2f HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 17:12:43 +05:30
Vivek Ratnavel Subramanian c1649bf1e1
HDDS-1226. Addendum. ozone-filesystem jar missing in hadoop classpath 
Closes #560
 2019-03-13 12:08:45 +01:00
Lokesh Jain 0d62753da9 HDDS-1128. Create stateful manager class for the pipeline creation scheduling. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 16:30:06 +05:30
Mukul Kumar Singh d602d462a6 HDDS-1256. Fix failure in TestKeyValueContainerCheck. Contributed by Supratim Deka. 2019-03-13 16:22:19 +05:30
Arpit Agarwal 24793d2d97
HDDS-1163. Basic framework for Ozone Data Scrubber. Contributed by Supratim Deka. 2019-03-13 04:32:39 +09:00
Márton Elek c449cdebe6
HDDS-807. Period should be an invalid character in bucket names. Contributed by Siddharth Wagle. 2019-03-12 19:14:55 +01:00
Ajay Yadav dcb0de848d
HDDS-1043. Enable token based authentication for S3 api 
Closes #561
 2019-03-12 18:47:06 +01:00
Xiaoyu Yao 2778259b88 HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593) 
Closes PR #593
 2019-03-12 09:34:18 -07:00
Mukul Kumar Singh daf4660eeb HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh. 2019-03-12 20:02:19 +05:30
Márton Elek d17e31e062
HDDS-1214. Enable tracing for the datanode read/write path. Contributed by Elek, Marton. 
Closes #550.
 2019-03-12 11:34:56 +01:00
Ajay Yadav 599e926d46 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar. 
This closes (#581)
 2019-03-11 15:01:04 -07:00
Shashikant Banerjee b4aa24d3c5 HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-03-11 23:15:49 +05:30
Bharat Viswanadham 68aa2ccd89
HDDS-1240. Fix check style issues caused by HDDS-1196. 2019-03-09 21:50:46 -08:00
Xiaoyu Yao 11db46956c
HDDS-1235. BaseHttpServer NPE is HTTP policy is HTTPS_ONLY. Contributed by Xiaoyu Yao. 
Closes #572
 2019-03-09 09:38:01 -08:00
Mukul Kumar Singh 8458ced11e HDDS-1238. Fix Ratis Snapshot creation error if the snapshot file already exists in ContainerStateMachine. Contributed by Shashikant Banerjee. 2019-03-09 22:46:20 +05:30
Bharat Viswanadham 341c076f05 HDDS-1196. Add a ReplicationStartTimer class. (#567) 2019-03-08 20:55:33 +05:30
Yiqun Lin fb851c9481 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan. 2019-03-08 16:59:41 +08:00
Ajay Yadav 064f38b3a5 HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547) 2019-03-07 14:41:52 -08:00
Vivek Ratnavel Subramanian 475011bbf8
HDDS-1226. ozone-filesystem jar missing in hadoop classpath 
Closes #560
 2019-03-07 14:09:13 +01:00
Hanisha Koneru bb12e81ec8
HDDS-1175. Serve read requests directly from RocksDB. (#557) 
HDDS-1175. Serve read requests directly from RocksDB.
 2019-03-06 19:44:55 -08:00
Vivek Ratnavel Subramanian a55fc36299 HDDS-1093. Configuration tab in OM/SCM ui is not displaying the correct values. 2019-03-06 17:43:57 -08:00
Mukul Kumar Singh 2c3ec37738 HDDS-1210. Ratis pipeline creation doesn't check raft client reply status during initialization. Contributed by Mukul Kumar Singh. 2019-03-06 21:38:21 +05:30
Lokesh Jain 129fd5dd18 HDDS-1208. ContainerStateMachine should set chunk data as state machine data for ratis. Contributed by Lokesh Jain. 2019-03-06 17:00:37 +05:30
Shashikant Banerjee 62e89dc275 HDDS-1184. Parallelization of write chunks in datanodes is broken. Contributed by Shashikant Banerjee. 2019-03-06 10:00:16 +05:30
Arpit Agarwal 1ad5bfc53f
HDDS-1188. Implement a skeleton patch for Recon server with initial set of interfaces. Contributed by Siddharth Wagle. 2019-03-05 15:54:29 -08:00
Elek, Márton 7f636b47b5 HDDS-919. Enable prometheus endpoints for Ozone datanodes (#502) 
Pull request from Elek, Márton. (#502)
 2019-03-05 12:04:57 -08:00
Bharat Viswanadham 313e8b9f13 HDDS-1193. Refactor ContainerChillModeRule and DatanodeChillMode rule. (#534) 
* HDDS-1193. Refactor ContainerChillModeRule and DatanodeChillMode rule.
 2019-03-05 11:46:36 -08:00
Arpit Agarwal df99137310
Merge pull request #555 from elek/HDDS-1219 
HDDS-1219. TestContainerActionsHandler.testCloseContainerAction has an intermittent failure
 2019-03-05 10:19:05 -08:00
Hanisha Koneru 12402b7a74 HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-05 09:24:22 -08:00
Shashikant Banerjee 5fcea54a50 HDDS-935. Avoid creating an already created container on a datanode in case of disk removal followed by datanode restart. Contributed by Shashikant Banerjee. 2019-03-05 22:09:46 +05:30
Márton Elek 296259e9b3
HDDS-1219. TestContainerActionsHandler.testCloseContainerAction has an intermittent failure. Contributed by Elek, Marton. 2019-03-05 13:45:47 +01:00
Elek, Márton 90c37ac40d HDDS-623. On SCM UI, Node Manager info is empty (#523) 
* HDDS-623. On SCM UI, Node Manager info is empty. Contributed by Elek, Marton.
 2019-03-04 14:35:00 -08:00
Anu Engineer 10b802b84b HDDS-1136 : Add metric counters to capture the RocksDB checkpointing statistics. 
Contributed by Aravindan Vijayan.
 2019-03-04 12:00:16 -08:00
Xiaoyu Yao eed8b794d8 HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao. (#545) 2019-03-04 10:37:26 -08:00
Hanisha Koneru b18c1c22ea
Revert "HDDS-1072. Implement RetryProxy and FailoverProxy for OM client." 
This reverts commit 8e1225991d.
 2019-03-04 16:58:59 +09:00
Ajay Yadav 9e53088493 Revert "HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contr…" (#544) 
* Revert "Fix checkstyle issue"

This reverts commit 78c8a3323e.

* Revert "HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao."

This reverts commit 6fef6fc5ee.
 2019-03-01 16:19:43 -08:00
Xiaoyu Yao 6fef6fc5ee HDDS-1183. Override getDelegationToken API for OzoneFileSystem. Contributed by Xiaoyu Yao. 2019-03-01 16:15:20 -08:00
Ajay Kumar 4a3cddea70 HDDS-134. SCM CA: OM sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. 2019-03-01 15:41:09 -08:00
Xiaoyu Yao eea40cb88a
HDDS-1204. Fix ClassNotFound issue with javax.xml.bind.DatatypeConverter used by DefaultProfile. Contributed by Xiaoyu Yao. 2019-03-01 10:49:39 -08:00
Shashikant Banerjee de1dae64f2 HDDS-726. Ozone Client should update SCM to move the container out of allocation path in case a write transaction fails. Contributed by Shashikant Banerjee. 2019-03-01 23:30:28 +05:30
Hanisha Koneru 8e1225991d HDDS-1072. Implement RetryProxy and FailoverProxy for OM client. 2019-03-01 00:27:39 -08:00
Bharat Viswanadham 77b23c816e
HDDS-1182. Pipeline Rule where at least one datanode is reported in the pipeline. 2019-02-28 21:41:36 -08:00
Anu Engineer eae3db9649 HDDS-1191. Replace Ozone Rest client with S3 client in smoketests and docs. 
Contributed Elek, Marton.
 2019-02-28 17:14:47 -08:00
Bharat Viswanadham eae8819fd2
HDDS-1187. Healthy pipeline Chill Mode rule to consider only pipelines with replication factor three. 2019-02-28 17:03:39 -08:00
Ajay Kumar cbf82fabf0 HDDS-1061. DelegationToken: Add certificate serial id to Ozone Delegation Token Identifier. Contributed by Ajay Kumar. 2019-02-27 17:00:42 -08:00
Ajay Kumar fef2e2fc88 HDDS-1176. Allow persisting X509CertImpl to SCM certificate table. Contributed by Xiaoyu Yao. 2019-02-27 10:40:41 -08:00
Bharat Viswanadham 6c8c422beb
HDDS-1178. Healthy pipeline Chill Mode Rule. 
Closes #518
 2019-02-27 14:26:40 +01:00
Xiaoyu Yao 9e0f3d1c52 HDDS-1038. Support Service Level Authorization for Ozone. Contributed by Xiaoyu Yao and Ajay Kumar. 2019-02-26 06:02:10 -08:00
Anu Engineer 9de34d2990 HDDS-1151. Propagate the tracing id in ScmBlockLocationProtocol. 
Contributed by Elek, Marton.
 2019-02-25 13:50:37 -08:00
Anu Engineer 92b1fdcece HDDS-1070. Adding Node and Pipeline related metrics in SCM. 
Contributed by Nandakumar.
 2019-02-24 15:39:41 -08:00
Anu Engineer 3aa0a57ea0 HDDS-1152. Add trace information for the client side of the datanode writes 
Contributed by Elek, Marton.
 2019-02-24 15:20:34 -08:00
Anu Engineer 5c1f946071 HDDS-1145. Add optional web server to the Ozone freon test tool. 
Contributed by Elek, Marton.
 2019-02-24 15:04:25 -08:00
Anu Engineer fc5c83b2ca HDDS-1090. Add a recipe to the ozone docs about using Spark, OzoneFS, and K8s. 
Contributed by Elek, Marton.
 2019-02-24 14:02:30 -08:00
Anu Engineer d82014b70a HDDS-1149. Change the default ozone.client.checksum.type. 
Contributed by Bharat Viswanadham.
 2019-02-24 13:22:36 -08:00
Anu Engineer 8387bbdfb5 HDDS-1154. Add tracing to the client side of StorageContainerLocationProtocol and OzoneManagerProtocol. 
Contributed by Elek, Marton.
 2019-02-24 13:13:37 -08:00
Anu Engineer 021f8deef0 HDDS-1168.Use random ports in TestBlockManager and TestDeletedBlockLog. 
Contributed by Nandakumar.
 2019-02-24 12:36:03 -08:00
Bharat Viswanadham 106bdc6c04
HDDS-1120. Add a config to disable checksum verification during read. 2019-02-23 21:37:55 -08:00
Anu Engineer 490206e4b4 HDDS-1155.Fix failing unit test methods of TestDeadNodeHandler. 
Contributed by Nandakumar.
 2019-02-23 21:13:47 -08:00
Anu Engineer 014e17af78 HDDS-1166. Fix checkstyle line length issues. 
Contributed by Nandakumar.
 2019-02-23 20:31:39 -08:00
Bharat Viswanadham 1b87668a3b HDDS-1165. Document generation in maven should be configured on execution level. Contributed by Anu Engineer. 2019-02-23 10:44:46 -08:00
Bharat Viswanadham 70579805c9 HDDS-1148. After allocating container, we are not adding to container DB. 2019-02-23 01:50:49 +05:30
Elek, Márton 28d0bf9f67 HDDS-1161. Disable failing test which are tracked by a separated jira. Contributed by Elek, Marton. 2019-02-21 13:57:05 -08:00
Bharat Viswanadham d33f0666f6
HDDS-1141. Update DBCheckpointSnapshot to DBCheckpoint. 
* HDDS-1141.Update DBCheckpointSnapshot to DBCheckpoint.

* fix test failures in TestOzoneConfigurationFields
 2019-02-21 13:29:10 -08:00
Nanda kumar 2bc3cfe28f HDDS-1126. Datanode is trying to qausi-close a container which is already closed. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-02-21 21:37:08 +05:30
Márton Elek 7c802c42dc
HDDS-1129. Fix findbug/checkstyle errors hdds projects. Contributed by Elek, Marton. 2019-02-21 14:04:18 +01:00
Arpit Agarwal 676a9cb888 HDDS-1053. Generate RaftGroupId from OMServiceID. Contributed by Aravindan Vijayan. 2019-02-20 12:57:49 -08:00
Xiaoyu Yao 1374f8f548 HDDS-1060. Add API to get OM certificate from SCM CA. Contributed by Ajay Kumar. 2019-02-20 11:35:49 -08:00
avijayanhwx e8d7e3b4e6 HDDS-1139 : Fix findbugs issues caused by HDDS-1085. Contributed by Aravindan Vijayan. 2019-02-19 16:32:47 -08:00
Anu Engineer 02d04bd107 HDDS-1121. Key read failure when data is written parallel in to Ozone. 
Contributed by Bharat Viswanadham.
 2019-02-19 11:07:17 -08:00
Xiaoyu Yao cf1a66d0da HDDS-1101. SCM CA: Write Certificate information to SCM Metadata. Contributed by Anu Engineer. 2019-02-19 10:42:56 -08:00
Anu Engineer 588b4c4d78 HDDS-1085. Create an OM API to serve snapshots to Recon server. 
Contributed by Aravindan Vijayan.
 2019-02-18 22:35:36 -08:00
Yiqun Lin f2fb6536dc HDDS-1106. Introduce queryMap in PipelineManager. Contributed by Lokesh Jain. 2019-02-18 22:35:23 +08:00
Anu Engineer 217bdbd940 HDDS-1116.Add java profiler servlet to the Ozone web servers. 
Contributed by Elek, Marton.
 2019-02-15 11:32:06 -08:00
Shashikant Banerjee de934ba2dc HDDS-1076. TestSCMNodeManager crashed the jvm. Contributed by Lokesh Jain. 2019-02-15 21:13:30 +05:30
Mukul Kumar Singh 9584b47e03 HDDS-1082. OutOfMemoryError because of memory leak in KeyInputStream. Contributed by Supratim Deka. 2019-02-15 20:09:15 +05:30
Shashikant Banerjee 8a426dc848 HDDS-1028. Improve logging in SCMPipelineManager. Contributed by Lokesh Jain. 2019-02-15 14:38:38 +05:30
Anu Engineer 75e15cc0c4 HDDS-1103.Fix rat/findbug/checkstyle errors in ozone/hdds projects. 
Contributed by Elek, Marton.
 2019-02-14 23:33:25 -08:00
Anu Engineer 0395f22145 HDDS-1068. Improve the error propagation for ozone sh. 
Contributed by Elek, Marton.
 2019-02-14 22:51:46 -08:00
Bharat Viswanadham 917ac9f108 HDDS-972. Add support for configuring multiple OMs. Contributed by Hanisha Koneru. 2019-02-12 22:07:27 -08:00
Nanda kumar 4f7d32edf8 HDDS-1081. CLOSING state containers should not be added to pipeline on SCM start. 
Contributed by Lokesh Jain.
 2019-02-12 14:39:08 +05:30
Nanda kumar a536eb5c41 HDDS-360. Use RocksDBStore and TableStore for SCM Metadata. 
Contributed by Anu Engineer.
 2019-02-12 14:26:19 +05:30
Ajay Kumar d48e61dd36 HDDS-1012. Add Default CertificateClient implementation. Contributed by Ajay Kumar 2019-02-11 18:59:47 -08:00
Márton Elek e7d1ae52d2 HDDS-1017. Use distributed tracing to indentify performance problems in Ozone. Contributed by Elek, Marton. 2019-02-11 12:18:26 +01:00
Shashikant Banerjee 965d26c9c7 HDDS-1026. Reads should fail over to alternate replica. Contributed by Shashikant Banerjee. 2019-02-10 10:53:16 +05:30
Yiqun Lin fb8c997a68 HDDS-1048. Remove SCMNodeStat from SCMNodeManager and use storage information from DatanodeInfo#StorageReportProto. Contributed by Nanda kumar. 2019-02-08 23:49:37 +08:00
Nanda kumar 0c1bc4dcee HDDS-981. Block allocation should involve pipeline selection and then container selection. 
Contributed by Lokesh Jain.
 2019-02-08 15:44:41 +05:30
Márton Elek a65aca2fef HDDS-922. Create isolated classloder to use ozonefs with any older hadoop versions. Contributed by Elek, Marton. 2019-02-07 17:02:03 +01:00
Mukul Kumar Singh 214112b2d7 HDDS-1010. ContainerSet#getContainerMap should be renamed. Contributed by Supratim Deka. 2019-02-07 18:06:23 +05:30