Commit Graph

925 Commits

Author SHA1 Message Date
Sammi Chen 493b0b5760 HDDS-1787. NPE thrown while trying to find DN closest to client. Contributed by Sammi Chen. (#1094) 2019-07-16 12:52:29 -07:00
Shashikant Banerjee d21eccf8ba HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. (#1084) 2019-07-16 18:01:13 +05:30
Nanda kumar 1502e6a22f HDDS-1756. DeleteContainerCommandHandler fails with NPE. Contributed by Nanda kumar(#1095). 2019-07-16 15:06:41 +05:30
Anu Engineer 47345e598e
Merge pull request #1058 from elek/HDDS-1763
HDDS-1763. Use vendor neutral s3 logo in ozone doc
2019-07-15 13:17:21 -07:00
Mukul Kumar Singh 0976f6fc30
HDDS-1766. ContainerStateMachine is unable to increment lastAppliedTermIndex. Contributed by Mukul Kumar Singh. (#1072) 2019-07-14 10:53:51 +05:30
Márton Elek 9119ed07ff
HDDS-1384. TestBlockOutputStreamWithFailures is failing
Closes #1029
2019-07-12 11:00:03 +02:00
Hrishikesh Gadre 14c43f85de HDDS-1752. ConcurrentModificationException while handling DeadNodeHandler event. (#1080) 2019-07-12 14:15:48 +05:30
supratimdeka 738fab3bff HDDS-1754. getContainerWithPipeline fails with PipelineNotFoundException. Contributed by Supratim Deka (#1081) 2019-07-12 10:31:48 +05:30
Nanda kumar 0f399b0d57
HDDS-1784. Missing HostName and IpAddress in the response of register command. 2019-07-11 19:01:06 +05:30
Hrishikesh Gadre acef5e0cec HDDS-1201. Reporting corrupted containers info to SCM (#1032) 2019-07-11 14:49:58 +05:30
Ajay Yadav cdb20adfcc HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. (#973) 2019-07-10 11:03:58 -07:00
supratimdeka e0fa1040af HDDS-1748. Error message for 3 way commit failure is not verbose. Contributed by Supratim Deka (#1051) 2019-07-10 16:45:55 +05:30
supratimdeka ac7a8accdf HDDS-1603. Handle Ratis Append Failure in Container State Machine. Contributed by Supratim Deka (#1019) 2019-07-10 07:49:36 +05:30
Xiaoyu Yao 030307226a
HDDS-1586. Allow Ozone RPC client to read with topology awareness. Contributed by Sammi Chen. 2019-07-09 14:43:55 -07:00
Mukul Kumar Singh 96d0555913 HDDS-1718. Increase Ratis Leader election timeout default. Contributed by Aravindan Vijayan & Siddharth Wagle. (#1062) 2019-07-09 23:17:50 +05:30
Vivek Ratnavel Subramanian 82d88a8d30 HDDS-1705. Recon: Add estimatedTotalCount to the response of containers and containers/{id} endpoints. Contributed by Vivek Ratnavel Subramanian. 2019-07-08 21:06:50 -07:00
Mukul Kumar Singh e5ffb88257 HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh. (#1050)
* HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh.
2019-07-08 20:24:12 -07:00
Lokesh Jain b5d30e4914 HDDS-1750. Add block allocation metrics for pipelines in SCM. Contributed by Lokesh Jain. (#1047) 2019-07-09 08:42:55 +05:30
Mukul Kumar Singh ce91d35b29
HDDS-1728. Add metrics for leader's latency in ContainerStateMachine. Contributed by Mukul Kumar Singh. (#1022) 2019-07-08 12:19:16 +05:30
Márton Elek 68f53f95ad
HDDS-1763. Use vendor neutral s3 logo in ozone doc. Contributed by Elek, Marton. 2019-07-04 17:14:19 +02:00
Mukul Kumar Singh 8b0d1adf31 HDDS-1616. ManagedChannel references are being leaked in while removing RaftGroup. Contributed by Mukul Kumar Singh. (#1039) 2019-07-02 14:34:49 -07:00
S O'Donnell f8d62a9c4c
HDDS-1258. Fix error propagation for SCM protocol
Closes #1001
2019-07-01 20:04:32 +02:00
dineshchitlangia 1f75660350 HDDS-373. Genconf tool must generate ozone-site.xml with sample values (#1025) 2019-07-01 08:58:47 -07:00
Mukul Kumar Singh 9df6275954 HDDS-1555. Disable install snapshot for ContainerStateMachine. Contributed by Siddharth Wagle. (#846) 2019-07-01 20:52:32 +05:30
dineshchitlangia da568996af HDDS-1733. Fix Ozone documentation (#1031) 2019-06-28 15:59:33 -07:00
Márton Elek 9fd3c702fc
HDDS-1639. Restructure documentation pages for better understanding
Closes #901
2019-06-28 19:51:30 +02:00
Bharat Viswanadham 4848280437
HDDS-1638. Implement Key Write Requests to use Cache and DoubleBuffer. (#956) 2019-06-26 18:04:12 -07:00
avijayanhwx 0a37ccf379 HDDS-1691 : RDBTable#isExist should use Rocksdb#keyMayExist (#1013) 2019-06-26 11:44:49 -07:00
Bharat Viswanadham a79bdf760e
HDDS-1727. Use generation of resourceName for locks in OzoneManagerLock. (#1014) 2019-06-25 17:12:45 -07:00
Bharat Viswanadham 049bb42760 HDDS-1723. Create new OzoneManagerLock class. (#1006)
* HDDS-1723. Create new OzoneManagerLock class.

* add logs

* fix exception message

* add license

* fix findbug

* throw exception when acquiring 2nd lock failed.

* fix review comments

* fix missed review comment

* fix checkstyle

* fix review comments
2019-06-25 08:48:03 -07:00
Elek, Márton 004207996c HDDS-1597. Remove hdds-server-scm dependency from ozone-common (#969)
* HDDS-1597. Remove hdds-server-scm dependency from ozone-common. Contributed by Elek, Marton.

* checkstyle fixes

* revert the import reorder of HddsUtil

* add javadoc

* switch back to the commons-lang2

* fix typo

* fix metrics core classpath problem (+rebase fix)
2019-06-24 11:56:04 -07:00
avijayanhwx 81d2f1b724 HDDS-1719 : Increase ratis log segment size to 1MB. (#1005)
* HDDS-1719 : Increase ratis log segment size to 1MB.

* HDDS-1719 : Increase ratis log segment size to 1MB.
2019-06-24 08:57:08 -07:00
Nanda kumar b28ddb22a5 HDDS-1706. Replication Manager thread running too frequently. Contributed by Nilotpal Nandi. 2019-06-22 16:15:45 +05:30
Hrishikesh Gadre 8194a1196e HDDS-1690. ContainerController should provide a way to retrieve containers per volume (#986) 2019-06-21 10:23:11 -07:00
S O'Donnell b95a58e231
HDDS-1674. Make ScmBlockLocationProtocol message type based.
Closes #984
2019-06-21 11:35:43 +02:00
supratimdeka 9d6842501c HDDS-1454. GC other system pause events can trigger pipeline destroy for all the nodes in the cluster. Contributed by Supratim Deka (#852) 2019-06-19 20:11:16 +05:30
Hanisha Koneru 2dfa932818
HDDS-1684. OM should create Ratis related dirs only if ratis is enabled (#965) 2019-06-18 16:08:48 -07:00
avijayanhwx 71edfce187 HDDS-1699. Update RocksDB version to 6.0.1 (#980) 2019-06-18 11:13:13 -07:00
Márton Elek e188bb12b0
HDDS-1694. TestNodeReportHandler is failing with NPE
Closes #978
2019-06-18 08:55:38 +02:00
Bharat Viswanadham ba681bb80e
HDDS-1692. RDBTable#iterator should disabled caching of the keys during iterator. (#975) 2019-06-16 22:00:39 -07:00
Elek, Márton e094b3b4cd HDDS-1680. Create missing parent directories during the creation of HddsVolume dirs (#961) 2019-06-13 16:18:15 -07:00
Bharat Viswanadham 88c53d516c
HDDS-1620. Implement Volume Write Requests to use Cache and DoubleBuffer. (#884) 2019-06-12 17:45:42 -07:00
Sammi Chen 1732312f45 HDDS-1663. Add datanode to network topology cluster during node regis… (#937) 2019-06-12 12:16:42 -07:00
Márton Elek 23c037906f
Revert "HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)"
This reverts commit e997f2a34a.
2019-06-12 08:52:16 +02:00
Elek, Márton e997f2a34a HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)
* HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject

* Remove Site improvements to display proposals
2019-06-11 11:20:21 -07:00
Márton Elek f918e3fe62
HDDS-1669. SCM startup is failing if network-topology-default.xml is part of a jar
Closes #946
2019-06-11 19:30:22 +02:00
Shweta Yakkali fcfe7a3cc0 HDDS-1655. Redundant toString() call for metaDataPath in KeyValueContainerCheck (#932) 2019-06-08 22:12:29 -07:00
Elek, Márton 46b23c11b0 HDDS-1636. Tracing id is not propagated via async datanode grpc call (#895) 2019-06-07 20:40:32 -07:00
S O'Donnell 85479577da
HDDS-1622. Use picocli for StorageContainerManager
Closes #918
2019-06-07 17:50:57 +02:00
Hanisha Koneru a91d24fea4
HDDS-1496. Support partial chunk reads and checksum verification (#804) 2019-06-06 19:44:40 -07:00
Hanisha Koneru 0b115b60b0 HDDS-1371. OMSnapshotProvider to download DB checkpoint from leader OM. (#703) 2019-06-06 19:42:34 -07:00
Sammi Chen 8ca58efeec HDDS-1490. Support configurable container placement policy through 'o… (#903) 2019-06-06 12:14:47 -07:00
Xiaoyu Yao c7e6f076df HDDS-1650. Fix Ozone tests leaking volume checker thread. Contributed by Xiaoyu Yao. (#915) 2019-06-06 23:50:04 +05:30
Xiaoyu Yao 76c0183ae3 HDDS-1652. HddsDispatcher should not shutdown volumeSet. Contributed by Xiaoyu Yao. (#916) 2019-06-06 23:47:59 +05:30
avijayanhwx fe069570d8 HDDS-1647 : Recon config tag does not show up on Ozone UI. (#914) 2019-06-06 23:43:29 +05:30
Shweta Yakkali c8276f3e76 HDDS-1201. Reporting Corruptions in Containers to SCM (#912) 2019-06-06 23:36:48 +05:30
Bharat Viswanadham 0b8a2608e0 HDDS-1605. Implement AuditLogging for OM HA Bucket write requests. (#867) 2019-06-06 10:13:36 -07:00
S O'Donnell 36757ad6a9
HDDS-1645. Change the version of Pico CLI to the latest 3.x release - 3.9.6
Closes #909
2019-06-06 16:00:00 +02:00
supratimdeka 9fded678ff HDDS-1621. writeData in ChunkUtils should not use AsynchronousFileChannel. Contributed by Supratim Deka (#917) 2019-06-06 18:53:37 +05:30
Sammi Chen 73954c1dd9 HDDS-1612. Add 'scmcli printTopology' shell command to print datanode topology. Contributed by Sammi Chen.(#910) 2019-06-05 20:13:39 -07:00
ChenSammi 0b1e288deb HDDS-1637. Fix random test failure TestSCMContainerPlacementRackAware. Contributed by Sammi Chen. (#904) 2019-06-05 09:09:36 -07:00
avijayanhwx 1a78794227 HDDS-1624 : Refactor operations inside the bucket lock in OM key write. (#882) 2019-06-04 15:41:37 -07:00
Elek, Márton 97607f3dce HDDS-1510. Classpath files are deployed to the maven repository as pom/jar files. Contributed by Elek, Marton. (#900) 2019-06-04 10:38:37 -07:00
bshashikant e4b4fec66f HDDS-1224. Restructure code to validate the response from server in the Read path (#806) 2019-06-04 10:37:02 -07:00
Mukul Kumar Singh 277e9a835b Opening of rocksDB in datanode fails with "No locks available"
Signed-off-by: Nanda kumar <nanda@apache.org>
2019-06-04 02:12:44 +05:30
avijayanhwx 21de9af903 HDDS-1625 : ConcurrentModificationException when SCM has containers of different owners. (#883) 2019-06-04 01:15:04 +05:30
Shashikant Banerjee f3271126fc HDDS-1558. IllegalArgumentException while processing container Reports.
Signed-off-by: Nanda kumar <nanda@apache.org>
2019-06-04 00:59:02 +05:30
Márton Elek 2a97a37d9e
Revert "HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)"
This reverts commit baee71551d.
2019-06-03 08:34:19 +02:00
Márton Elek 1ae062c818
HDDS-1382. Create customized CSI server for Ozone
Closes #693
2019-05-31 13:35:58 +02:00
Bharat Viswanadham c965f7f499
HDDS-1551. Implement Bucket Write Requests to use Cache and DoubleBuffer. (#850) 2019-05-30 15:28:15 -07:00
Elek, Márton baee71551d HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860) 2019-05-30 09:20:16 -07:00
avijayanhwx 33419a980a HDDS-1568 : Add RocksDB metrics to OM. Contributed by Aravindan Vijayan 2019-05-30 08:48:14 -07:00
dineshchitlangia 0b907bc0e8 HDDS-1581. Atleast one of the metadata dir config property must be tagged as REQUIRED. 2019-05-30 08:39:19 -07:00
Shashikant Banerjee 18c1eebc08 HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833). 2019-05-30 16:17:45 +05:30
Bharat Viswanadham 751f0df710 HDDS-1231. Add ChillMode metrics. Contributed by Bharat Viswanadham. 2019-05-29 20:55:44 +02:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
2019-05-29 17:25:13 +09:00
supratimdeka 346c2b7980 HDDS-1559. Include committedBytes to determine Out of Space in VolumeChoosingPolicy. Contributed by Supratim Deka (#841) 2019-05-28 20:48:14 -07:00
Xiaoyu Yao 7f2e87a419
HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866) 2019-05-28 18:39:23 -07:00
Elek, Márton 79d14d0d42 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862) 2019-05-28 14:40:45 -07:00
Bharat Viswanadham d8b18e82f0
HDDS-1602. Fix TestContainerPersistence#testDeleteBlockTwice. (#858) 2019-05-28 11:27:16 -07:00
Shweta Yakkali d1ec1c52e5 HDDS-1580. Obtain Handler reference in ContainerScrubber (#842) 2019-05-28 11:14:24 -07:00
Mukul Kumar Singh 9f0d3412f8
HDDS-1533. JVM exit on TestHddsDatanodeService
Closes #859
2019-05-28 14:18:25 +02:00
Bharat Viswanadham b70d1be685 HDDS-1559. Fix TestReplicationManager. Contributed by Bharat Viswanadham. (#856) 2019-05-27 17:52:09 -07:00
Elek, Márton ec92ca6575 HDDS-1598. Fix Ozone checkstyle issues on trunk. Contributed by Elek, Marton. (#854) 2019-05-27 14:40:51 -07:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
avijayanhwx 4b099b8b89 HDDS-1501 : Create a Recon task interface to update internal DB on updates from OM. (#819) 2019-05-23 15:34:55 -07:00
Arpit Agarwal b4b9120e28
HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka (#832)
* HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka

* Fixed checkstyle issues.

* verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion

* fixed merge error. adapted to new signature of BlockUtils getDB

* fixed checkstyle issue post merge
2019-05-23 10:42:08 -07:00
Xiaoyu Yao 20a4ec351c HDDS-700. Support rack awared node placement policy based on network topology. Contributed by Sammi Chen. 2019-05-23 10:40:15 -07:00
sdeka ca93760504 fixed checkstyle issue post merge 2019-05-23 20:49:52 +05:30
sdeka 72bef0f6cb fixed merge error. adapted to new signature of BlockUtils getDB 2019-05-23 11:36:40 +05:30
supratimdeka 9da62f33be
Merge branch 'trunk' into HDDS-1535 2019-05-23 09:51:33 +05:30
Ajay Yadav 9c61494c02 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834) 2019-05-22 11:47:32 -07:00
Shashikant Banerjee a315913c48 HDDS-1517. AllocateBlock call fails with ContainerNotFoundException (#826). Contributed by Shashikant Banerjee. 2019-05-22 17:24:27 +05:30
Mukul Kumar Singh 2fc6f8599a HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825) 2019-05-22 17:18:40 +05:30
sdeka 456bb8a3e0 verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion 2019-05-22 09:34:21 +05:30
Bharat Viswanadham 77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Lokesh Jain c1d7d68c78
HDDS-1461. Optimize listStatus api in OzoneFileSystem (#782) 2019-05-21 14:47:48 +05:30
avijayanhwx f63300228e HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799)
* HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected.

* HDDS-1451 : Address review comments.
2019-05-20 22:02:56 +02:00
sdeka 64c39856d2 Fixed checkstyle issues. 2019-05-20 08:13:37 +05:30
Bharat Viswanadham 0d1d7c86ec HDDS-1499. OzoneManager Cache. (#798) 2019-05-19 19:23:02 -07:00
sdeka 869a1ab41a HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka 2019-05-18 16:16:33 +05:30
Arpit Agarwal 732133cb2a HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725)
* HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit Agarwal.

Change-Id: I6db6bdd19839a45e5341ed7e745cd38f68af8378

* Suppress spurious findbugs warning.

* Remove log file that got committed in error
2019-05-18 11:47:19 +02:00
Siddharth c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle.
This closes #822.
2019-05-16 15:13:10 -07:00
Márton Elek 03ea8ea92e
HDDS-1297. Fix IllegalArgumentException thrown with MiniOzoneCluster Initialization. Contributed by Yiqun Lin. 2019-05-16 17:04:46 +02:00
Márton Elek 9248b7d58b
HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. 2019-05-16 17:04:46 +02:00
Shashikant Banerjee e66ecc564a HDDS-1531. Disable the sync flag by default during chunk writes in Datanode (#820). Contributed by Shashikant Banerjee. 2019-05-16 18:38:52 +05:30
supratimdeka 9569015802 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812) 2019-05-15 10:26:01 -07:00
Hanisha Koneru 02c9efcb81
HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795) 2019-05-13 20:49:52 -07:00
Vivek Ratnavel Subramanian b832e174b0 HDDS-1474. "ozone.scm.datanode.id" config should take path for a dir and not a file (#809) 2019-05-09 17:11:09 -07:00
Hanisha Koneru 2d31ccc1b3
Revert "HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792)" (#808)
This reverts commit 18cca583ac.
2019-05-09 11:18:46 -07:00
Vivek Ratnavel Subramanian 18cca583ac HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792) 2019-05-09 10:23:44 -07:00
avijayanhwx eb9c8900bc HDDS-1475 : Fix OzoneContainer start method. (#788) 2019-05-07 14:09:41 -07:00
Shashikant Banerjee 7f0e2c67e0 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee 2019-05-08 01:00:10 +05:30
Shashikant Banerjee 01451a57de Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)"
This reverts commit d525633619.
2019-05-08 00:49:15 +05:30
bshashikant d525633619
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749).
2019-05-08 00:38:53 +05:30
avijayanhwx f194540520 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786) 2019-05-04 00:19:00 +05:30
Siddharth 1df679985b HDDS-1473. DataNode ID file should be human readable. (#781) 2019-05-02 13:59:14 -07:00
Márton Elek e2f0f72677
HDDS-1469. Generate default configuration fragments based on annotations
Closes #773
2019-05-02 12:14:43 +02:00
Vivek Ratnavel Subramanian 3cb1d09b2e
HDDS-1479. Update S3.md documentation
Closes #791
2019-05-02 11:49:37 +02:00
Márton Elek a2887f5c23
HDDS-1468. Inject configuration values to Java objects
Closes #772
2019-05-02 11:35:01 +02:00
Ajay Kumar 382a962d8d
Revert "HDDS-973. HDDS/Ozone fail to build on Windows."
This reverts commit 5dae1a0c66.
2019-04-29 19:46:20 -07:00
Elek, Márton 59816dff94 HDDS-1462. Fix content and format of Ozone documentation. Contributed by Elek, Marton. (#767) 2019-04-29 13:28:19 -07:00
Siddharth 5bca062d0e HDDS-1476. Fix logIfNeeded logic in EndPointStateMachine. (#779) 2019-04-29 12:05:38 -07:00
Xiaoyu Yao a96d16a843
HDDS-1455. Inconsistent naming convention with Ozone Kerberos configuration. Contributed by Xiaoyu Yao. (#762) 2019-04-29 11:18:11 -07:00
Ajay Yadav b434f558ca
HDDS-1471. Update ratis dependency to 0.3.0. Contributed by Ajay Kumar. (#777) 2019-04-28 20:57:59 -07:00
Bharat Viswanadham 43b2a4b77b
HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769) 2019-04-26 14:25:34 -07:00
Hanisha Koneru 37582705fa
HDDS-1403. KeyOutputStream writes fails after max retries while writing to a closed container (#753) 2019-04-26 10:39:03 -07:00
Eric Yang e95c055d91
HDDS-1457. Sequence of configuring Ozone File System instruction needs some work. Contributed by Eric Yang. 2019-04-24 15:52:51 +02:00
Kitti Nanasi 3f787cd506
HDDS-1192. Support -conf command line argument in GenericCli
Closes #713
2019-04-24 15:34:49 +02:00
Xiaoyu Yao 64f30da428 HDDS-1450. Fix nightly run failures after HDDS-976. Contributed by Xiaoyu Yao. (#757) 2019-04-23 21:38:23 -07:00
Siddharth 59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Nanda kumar 7e1f8d3a1b
HDDS-1368. Cleanup old ReplicationManager code from SCM. 2019-04-23 17:35:39 +05:30
Chen, Junjie 518f47bf9b HDDS-976: Parse network topology from yaml file. Contributed by Junjie Chen. (#661)
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
2019-04-18 16:53:25 -07:00
Márton Elek 8aea5e4cb6
HDDS-1433. Rename GetScmInfoRespsonseProto to GetScmInfoResponseProto due to typos. Contributed by Wanqiang Ji. 2019-04-18 08:53:08 +02:00
Shashikant Banerjee d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee 082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)"
This reverts commit df2ae27f3e.
2019-04-17 14:38:08 +05:30
Shashikant Banerjee df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373"
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
2019-04-17 14:17:54 +05:30
bshashikant 4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
2019-04-17 14:12:52 +05:30
Doroszlai, Attila 0a87fd1464 HDDS-1434. TestDatanodeStateMachine is flaky (#740) 2019-04-16 19:13:01 -07:00
Hanisha Koneru e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Bharat Viswanadham 3ac3e50b20
HDDS-1374. ContainerStateMap cannot find container while allocating blocks. (#735) 2019-04-16 13:49:29 -07:00
Arpit Agarwal ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Arpit Agarwal 62f4808617
HDDS-1428. Remove benign warning in handleCreateContainer. Contributed by Siddharth Wagle. 2019-04-12 15:08:01 -07:00
Bharat Viswanadham a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher
Closes #723
2019-04-12 12:21:10 +02:00
Arpit Agarwal a9b46c58fe
HDDS-1421. Avoid unnecessary object allocations in TracingUtil
Closes #722
2019-04-12 11:52:31 +02:00
Doroszlai, Attila 3b08ac4659 HDDS-1363. ozone.metadata.dirs doesn't pick multiple dirs (#691) 2019-04-12 15:00:08 +05:30
Nanda kumar c298415eb5
HDDS-1404. Fix typos in HDDS. Contributed by bianqi. 2019-04-12 14:12:10 +05:30
Shashikant Banerjee 22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Nanda kumar e9c4109004 HDDS-1417. After successfully importing a container, datanode should delete the container tar.gz file from working directory. (#721) 2019-04-10 16:55:37 -07:00
Bharat Viswanadham 0e770a6539
HDDS-1370. Command Execution in Datanode fails because of NPE (#715) 2019-04-10 10:25:28 -07:00
Márton Elek dfb518bbf5
HDDS-1397. Avoid the usage of signal handlers in datanodes of the MiniOzoneClusters.
Closes #701
2019-04-10 18:35:36 +02:00
Mukul Kumar Singh df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Mukul Kumar Singh 312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Nanda kumar 73f43ac2dc
HDDS-1372. getContainerWithPipeline for a standalone pipeline fails with ConcurrentModificationException. (#682) 2019-04-09 15:10:07 +05:30
Shashikant Banerjee 2d4f6b6daa HDDS-372. There are three buffer copies in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-04-09 10:27:58 +05:30
Anu Engineer dc013f7a35 HDDS-1329. Update documentation for Ozone-0.4.0 release.
Contributed By Ajay Kumar.
2019-04-05 14:14:42 -07:00
Doroszlai, Attila 274ab4e86c HDDS-1332. Attempt to fix flaky test testStartStopDatanodeStateMachine (#697) 2019-04-05 13:21:58 -07:00
Nanda kumar 5750bb94ed HDDS-1389. Fix testSCMChillModeRestrictedOp. (#696) 2019-04-05 10:42:00 -07:00
Ajay Kumar 0356b3699d HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653)
(cherry picked from commit 7ec6a31eb3)
2019-04-05 09:29:21 -07:00
avijayanhwx 993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Shashikant Banerjee 33e159bf8d HDDS-1349. Remove watchClient from XceiverClientRatis. Contributed by Shashikant Banerjee. 2019-04-04 16:35:37 +05:30
Nanda kumar 48a58bce37
HDDS-1207. Refactor Container Report Processing logic and plugin new Replication Manager. (#662) 2019-04-04 16:32:59 +05:30
Yiqun Lin a92806d05a HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle. 2019-04-04 17:33:37 +08:00
Hanisha Koneru f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Yiqun Lin f96fb05a2b HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka. 2019-04-03 14:01:30 +08:00
Kitti Nanasi aeb43dfba3
HDDS-1153. Make tracing instrumentation configurable
Closes #673
2019-04-02 14:21:47 +02:00
Yiqun Lin d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Doroszlai, Attila 36756703f0 HDDS-1322. Hugo errors when building Ozone (#671) 2019-04-01 08:52:05 -07:00
Yiqun Lin 8bfef21efa HDDS-1360. Invalid metric type due to fully qualified class name. Contributed by Doroszlai, Attila. 2019-04-01 19:26:44 +08:00
Shweta Yakkali 82769ba1b8 HDDS-1352. Remove unused call in TestStorageContainerManagerHttpServer.
Signed-off-by: Nanda kumar <nanda@apache.org>
2019-03-29 17:26:48 +05:30
Xiaoyu Yao d7a2f9456b HDDS-1350. Fix checkstyle issue in TestDatanodeStateMachine. Contributed by Xiaoyu Yao. (#656) 2019-03-28 14:50:34 -07:00
nilotpalnandi 4cceeb2682 HDDS-1309 . change logging from warn to debug in XceiverClient. Contributed by Nilotpal Nandi. (#638) 2019-03-28 12:22:44 -07:00
Shashikant Banerjee ac4010bb22 HDDS-1293. ExcludeList#getProtoBuf throws ArrayIndexOutOfBoundsException. Contributed by Shashikant Banerjee. 2019-03-29 00:43:28 +05:30
Xiaoyu Yao ca5e4ce036
HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao.
This closes #641
2019-03-28 12:00:58 -07:00
Arpit Agarwal b4ed81c4e6 HDDS-1332. Add some logging for flaky test testStartStopDatanodeStateMachine. Contributed by Arpit Agarwal. (#649) 2019-03-27 20:53:30 +05:30
Sandeep Nemuri 5c0a81ad3c HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails.
Signed-off-by: Nanda kumar <nanda@apache.org>
2019-03-26 15:44:18 +05:30
Bharat Viswanadham 8739693514 HDDS-1217. Refactor ChillMode rules and chillmode manager. (#558) 2019-03-26 01:41:03 +05:30
Yiqun Lin e5d72f504e HDDS-1234. Iterate the OM DB snapshot and populate the recon container DB. Contributed by Aravindan Vijayan. 2019-03-25 22:52:02 +08:00
Shashikant Banerjee d4e4a7d456 HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee. 2019-03-25 15:41:20 +05:30
Bharat Viswanadham 128dd91e10
HDDS-1331. In DatanodeStateMachine join check for not null. 2019-03-23 09:43:49 -07:00
Bharat Viswanadham 43e421afef
HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-22 16:44:12 -07:00
Ajay Kumar 771ea6b5e7 Revert "HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri."
This reverts commit efad5717ec.
2019-03-22 16:20:44 -07:00
Ajay Kumar efad5717ec HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri. 2019-03-22 15:53:12 -07:00
Vivek Ratnavel Subramanian 73f7b04e2b HDDS-1302. Fix SCM CLI does not list container with id 1. 2019-03-22 15:15:01 -07:00
Arpit Agarwal f854a89190
HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED containers. Contributed by Nanda kumar. (#620) 2019-03-22 14:36:29 -07:00
Arpit Agarwal 90afc9ab03
HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-21 14:24:02 -07:00
Yiqun Lin 60cdd4cac1 HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan. 2019-03-21 11:02:29 +08:00
Mukul Kumar Singh 072750c00a HDDS-1306. TestContainerStateManagerIntegration fails in Ratis shutdown. Contributed by Lokesh Jain. 2019-03-20 17:14:19 +05:30
Ajay Yadav f10d493325
HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 2019-03-18 23:08:17 -07:00
Arpit Agarwal ae3a2c3851
Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)"
This reverts commit 44b8451821.
2019-03-18 14:21:57 -07:00
Sammi Chen 4d2a116d6e HDDS-699. Detect Ozone Network topology. Contributed by Sammi Chen. 2019-03-17 17:08:29 +08:00
Bharat Viswanadham 926d548caa
HDDS-1281. Fix the findbug issue caused by HDDS-1163. Contributed by Aravindan Vijayan. 2019-03-15 21:06:44 -07:00
Vivek Ratnavel Subramanian af2dfc9f3d HDDS-1263. SCM CLI does not list container with id 1. 2019-03-15 15:39:30 -07:00
Elek, Márton 44b8451821 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608) 2019-03-15 14:51:37 -07:00
Shashikant Banerjee 155ab6d5d8 HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee. 2019-03-15 15:40:59 +05:30
Vivek Ratnavel Subramanian 9001508485 HDDS-761. Create S3 subcommand to run S3 related operations.
Closes #579
2019-03-14 21:16:59 -07:00