Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
22,966 Commits 326 Branches 371 Tags 1.5 GiB
Commit Graph

901 Commits

Author SHA1 Message Date
dineshchitlangia da568996af HDDS-1733. Fix Ozone documentation (#1031) 2019-06-28 15:59:33 -07:00
Márton Elek 9fd3c702fc
HDDS-1639. Restructure documentation pages for better understanding 
Closes #901
 2019-06-28 19:51:30 +02:00
Bharat Viswanadham 4848280437
HDDS-1638. Implement Key Write Requests to use Cache and DoubleBuffer. (#956) 2019-06-26 18:04:12 -07:00
avijayanhwx 0a37ccf379 HDDS-1691 : RDBTable#isExist should use Rocksdb#keyMayExist (#1013) 2019-06-26 11:44:49 -07:00
Bharat Viswanadham a79bdf760e
HDDS-1727. Use generation of resourceName for locks in OzoneManagerLock. (#1014) 2019-06-25 17:12:45 -07:00
Bharat Viswanadham 049bb42760 HDDS-1723. Create new OzoneManagerLock class. (#1006) 
* HDDS-1723. Create new OzoneManagerLock class.

* add logs

* fix exception message

* add license

* fix findbug

* throw exception when acquiring 2nd lock failed.

* fix review comments

* fix missed review comment

* fix checkstyle

* fix review comments
 2019-06-25 08:48:03 -07:00
Elek, Márton 004207996c HDDS-1597. Remove hdds-server-scm dependency from ozone-common (#969) 
* HDDS-1597. Remove hdds-server-scm dependency from ozone-common. Contributed by Elek, Marton.

* checkstyle fixes

* revert the import reorder of HddsUtil

* add javadoc

* switch back to the commons-lang2

* fix typo

* fix metrics core classpath problem (+rebase fix)
 2019-06-24 11:56:04 -07:00
avijayanhwx 81d2f1b724 HDDS-1719 : Increase ratis log segment size to 1MB. (#1005) 
* HDDS-1719 : Increase ratis log segment size to 1MB.

* HDDS-1719 : Increase ratis log segment size to 1MB.
 2019-06-24 08:57:08 -07:00
Nanda kumar b28ddb22a5 HDDS-1706. Replication Manager thread running too frequently. Contributed by Nilotpal Nandi. 2019-06-22 16:15:45 +05:30
Hrishikesh Gadre 8194a1196e HDDS-1690. ContainerController should provide a way to retrieve containers per volume (#986) 2019-06-21 10:23:11 -07:00
S O'Donnell b95a58e231
HDDS-1674. Make ScmBlockLocationProtocol message type based. 
Closes #984
 2019-06-21 11:35:43 +02:00
supratimdeka 9d6842501c HDDS-1454. GC other system pause events can trigger pipeline destroy for all the nodes in the cluster. Contributed by Supratim Deka (#852) 2019-06-19 20:11:16 +05:30
Hanisha Koneru 2dfa932818
HDDS-1684. OM should create Ratis related dirs only if ratis is enabled (#965) 2019-06-18 16:08:48 -07:00
avijayanhwx 71edfce187 HDDS-1699. Update RocksDB version to 6.0.1 (#980) 2019-06-18 11:13:13 -07:00
Márton Elek e188bb12b0
HDDS-1694. TestNodeReportHandler is failing with NPE 
Closes #978
 2019-06-18 08:55:38 +02:00
Bharat Viswanadham ba681bb80e
HDDS-1692. RDBTable#iterator should disabled caching of the keys during iterator. (#975) 2019-06-16 22:00:39 -07:00
Elek, Márton e094b3b4cd HDDS-1680. Create missing parent directories during the creation of HddsVolume dirs (#961) 2019-06-13 16:18:15 -07:00
Bharat Viswanadham 88c53d516c
HDDS-1620. Implement Volume Write Requests to use Cache and DoubleBuffer. (#884) 2019-06-12 17:45:42 -07:00
Sammi Chen 1732312f45 HDDS-1663. Add datanode to network topology cluster during node regis… (#937) 2019-06-12 12:16:42 -07:00
Márton Elek 23c037906f
Revert "HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922)" 
This reverts commit e997f2a34a.
 2019-06-12 08:52:16 +02:00
Elek, Márton e997f2a34a HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject (#922) 
* HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject

* Remove Site improvements to display proposals
 2019-06-11 11:20:21 -07:00
Márton Elek f918e3fe62
HDDS-1669. SCM startup is failing if network-topology-default.xml is part of a jar 
Closes #946
 2019-06-11 19:30:22 +02:00
Shweta Yakkali fcfe7a3cc0 HDDS-1655. Redundant toString() call for metaDataPath in KeyValueContainerCheck (#932) 2019-06-08 22:12:29 -07:00
Elek, Márton 46b23c11b0 HDDS-1636. Tracing id is not propagated via async datanode grpc call (#895) 2019-06-07 20:40:32 -07:00
S O'Donnell 85479577da
HDDS-1622. Use picocli for StorageContainerManager 
Closes #918
 2019-06-07 17:50:57 +02:00
Hanisha Koneru a91d24fea4
HDDS-1496. Support partial chunk reads and checksum verification (#804) 2019-06-06 19:44:40 -07:00
Hanisha Koneru 0b115b60b0 HDDS-1371. OMSnapshotProvider to download DB checkpoint from leader OM. (#703) 2019-06-06 19:42:34 -07:00
Sammi Chen 8ca58efeec HDDS-1490. Support configurable container placement policy through 'o… (#903) 2019-06-06 12:14:47 -07:00
Xiaoyu Yao c7e6f076df HDDS-1650. Fix Ozone tests leaking volume checker thread. Contributed by Xiaoyu Yao. (#915) 2019-06-06 23:50:04 +05:30
Xiaoyu Yao 76c0183ae3 HDDS-1652. HddsDispatcher should not shutdown volumeSet. Contributed by Xiaoyu Yao. (#916) 2019-06-06 23:47:59 +05:30
avijayanhwx fe069570d8 HDDS-1647 : Recon config tag does not show up on Ozone UI. (#914) 2019-06-06 23:43:29 +05:30
Shweta Yakkali c8276f3e76 HDDS-1201. Reporting Corruptions in Containers to SCM (#912) 2019-06-06 23:36:48 +05:30
Bharat Viswanadham 0b8a2608e0 HDDS-1605. Implement AuditLogging for OM HA Bucket write requests. (#867) 2019-06-06 10:13:36 -07:00
S O'Donnell 36757ad6a9
HDDS-1645. Change the version of Pico CLI to the latest 3.x release - 3.9.6 
Closes #909
 2019-06-06 16:00:00 +02:00
supratimdeka 9fded678ff HDDS-1621. writeData in ChunkUtils should not use AsynchronousFileChannel. Contributed by Supratim Deka (#917) 2019-06-06 18:53:37 +05:30
Sammi Chen 73954c1dd9 HDDS-1612. Add 'scmcli printTopology' shell command to print datanode topology. Contributed by Sammi Chen.(#910) 2019-06-05 20:13:39 -07:00
ChenSammi 0b1e288deb HDDS-1637. Fix random test failure TestSCMContainerPlacementRackAware. Contributed by Sammi Chen. (#904) 2019-06-05 09:09:36 -07:00
avijayanhwx 1a78794227 HDDS-1624 : Refactor operations inside the bucket lock in OM key write. (#882) 2019-06-04 15:41:37 -07:00
Elek, Márton 97607f3dce HDDS-1510. Classpath files are deployed to the maven repository as pom/jar files. Contributed by Elek, Marton. (#900) 2019-06-04 10:38:37 -07:00
bshashikant e4b4fec66f HDDS-1224. Restructure code to validate the response from server in the Read path (#806) 2019-06-04 10:37:02 -07:00
Mukul Kumar Singh 277e9a835b Opening of rocksDB in datanode fails with "No locks available" 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-06-04 02:12:44 +05:30
avijayanhwx 21de9af903 HDDS-1625 : ConcurrentModificationException when SCM has containers of different owners. (#883) 2019-06-04 01:15:04 +05:30
Shashikant Banerjee f3271126fc HDDS-1558. IllegalArgumentException while processing container Reports. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-06-04 00:59:02 +05:30
Márton Elek 2a97a37d9e
Revert "HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860)" 
This reverts commit baee71551d.
 2019-06-03 08:34:19 +02:00
Márton Elek 1ae062c818
HDDS-1382. Create customized CSI server for Ozone 
Closes #693
 2019-05-31 13:35:58 +02:00
Bharat Viswanadham c965f7f499
HDDS-1551. Implement Bucket Write Requests to use Cache and DoubleBuffer. (#850) 2019-05-30 15:28:15 -07:00
Elek, Márton baee71551d HDDS-1597. Remove hdds-server-scm dependency from ozone-common. (#860) 2019-05-30 09:20:16 -07:00
avijayanhwx 33419a980a HDDS-1568 : Add RocksDB metrics to OM. Contributed by Aravindan Vijayan 2019-05-30 08:48:14 -07:00
dineshchitlangia 0b907bc0e8 HDDS-1581. Atleast one of the metadata dir config property must be tagged as REQUIRED. 2019-05-30 08:39:19 -07:00
Shashikant Banerjee 18c1eebc08 HDDS-1502. Add metrics for Ozone Ratis performance.Contributed by Shashikant Banerjee(#833). 2019-05-30 16:17:45 +05:30
Bharat Viswanadham 751f0df710 HDDS-1231. Add ChillMode metrics. Contributed by Bharat Viswanadham. 2019-05-29 20:55:44 +02:00
Akira Ajisaka afd844059c HADOOP-16331. Fix ASF License check in pom.xml 
Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
 2019-05-29 17:25:13 +09:00
supratimdeka 346c2b7980 HDDS-1559. Include committedBytes to determine Out of Space in VolumeChoosingPolicy. Contributed by Supratim Deka (#841) 2019-05-28 20:48:14 -07:00
Xiaoyu Yao 7f2e87a419
HDDS-1604. ContainerReader#initializeUsedBytes leaks DB reference. Co… (#866) 2019-05-28 18:39:23 -07:00
Elek, Márton 79d14d0d42 HDDS-1341. TestContainerReplication#testContainerReplication fails intermittently. Contributed by Elek, Marton. (#862) 2019-05-28 14:40:45 -07:00
Bharat Viswanadham d8b18e82f0
HDDS-1602. Fix TestContainerPersistence#testDeleteBlockTwice. (#858) 2019-05-28 11:27:16 -07:00
Shweta Yakkali d1ec1c52e5 HDDS-1580. Obtain Handler reference in ContainerScrubber (#842) 2019-05-28 11:14:24 -07:00
Mukul Kumar Singh 9f0d3412f8
HDDS-1533. JVM exit on TestHddsDatanodeService 
Closes #859
 2019-05-28 14:18:25 +02:00
Bharat Viswanadham b70d1be685 HDDS-1559. Fix TestReplicationManager. Contributed by Bharat Viswanadham. (#856) 2019-05-27 17:52:09 -07:00
Elek, Márton ec92ca6575 HDDS-1598. Fix Ozone checkstyle issues on trunk. Contributed by Elek, Marton. (#854) 2019-05-27 14:40:51 -07:00
Akira Ajisaka 9f933e6446
HADOOP-16323. https everywhere in Maven settings. 2019-05-27 15:24:59 +09:00
avijayanhwx 4b099b8b89 HDDS-1501 : Create a Recon task interface to update internal DB on updates from OM. (#819) 2019-05-23 15:34:55 -07:00
Arpit Agarwal b4b9120e28
HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka (#832) 
* HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka

* Fixed checkstyle issues.

* verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion

* fixed merge error. adapted to new signature of BlockUtils getDB

* fixed checkstyle issue post merge
 2019-05-23 10:42:08 -07:00
Xiaoyu Yao 20a4ec351c HDDS-700. Support rack awared node placement policy based on network topology. Contributed by Sammi Chen. 2019-05-23 10:40:15 -07:00
sdeka ca93760504 fixed checkstyle issue post merge 2019-05-23 20:49:52 +05:30
sdeka 72bef0f6cb fixed merge error. adapted to new signature of BlockUtils getDB 2019-05-23 11:36:40 +05:30
supratimdeka 9da62f33be
Merge branch 'trunk' into HDDS-1535 2019-05-23 09:51:33 +05:30
Ajay Yadav 9c61494c02 HDDS-1065. OM and DN should persist SCM certificate as the trust root. Contributed by Ajay Kumar. (#834) 2019-05-22 11:47:32 -07:00
Shashikant Banerjee a315913c48 HDDS-1517. AllocateBlock call fails with ContainerNotFoundException (#826). Contributed by Shashikant Banerjee. 2019-05-22 17:24:27 +05:30
Mukul Kumar Singh 2fc6f8599a HDDS-1449. JVM Exit in datanode while committing a key. Contributed by Mukul Kumar Singh. (#825) 2019-05-22 17:18:40 +05:30
sdeka 456bb8a3e0 verifyContainerData also does fixup, renamed. Added a Javadoc comment, both as per review discussion 2019-05-22 09:34:21 +05:30
Bharat Viswanadham 77c49f2963
HDDS-1406. Avoid usage of commonPool in RatisPipelineUtils. (#714) 2019-05-21 18:41:30 -07:00
Ajay Yadav ef1cc725b8
HDDS-1538. Update ozone protobuf message for ACLs. Contributed by Ajay Kumar. (#828) 2019-05-21 15:53:40 -07:00
Lokesh Jain c1d7d68c78
HDDS-1461. Optimize listStatus api in OzoneFileSystem (#782) 2019-05-21 14:47:48 +05:30
avijayanhwx f63300228e HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected. (#799) 
* HDDS-1451 : SCMBlockManager findPipeline and createPipeline are not lock protected.

* HDDS-1451 : Address review comments.
 2019-05-20 22:02:56 +02:00
sdeka 64c39856d2 Fixed checkstyle issues. 2019-05-20 08:13:37 +05:30
Bharat Viswanadham 0d1d7c86ec HDDS-1499. OzoneManager Cache. (#798) 2019-05-19 19:23:02 -07:00
sdeka 869a1ab41a HDDS-1535. Space tracking for Open Containers : Handle Node Startup. Contributed by Supratim Deka 2019-05-18 16:16:33 +05:30
Arpit Agarwal 732133cb2a HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit A… (#725) 
* HDDS-1422. Exception during DataNode shutdown. Contributed by Arpit Agarwal.

Change-Id: I6db6bdd19839a45e5341ed7e745cd38f68af8378

* Suppress spurious findbugs warning.

* Remove log file that got committed in error
 2019-05-18 11:47:19 +02:00
Siddharth c183bd8e20 HDDS-1527. HDDS Datanode start fails due to datanode.id file read error. Contributed by Siddharth Wagle. 
This closes #822.
 2019-05-16 15:13:10 -07:00
Márton Elek 03ea8ea92e
HDDS-1297. Fix IllegalArgumentException thrown with MiniOzoneCluster Initialization. Contributed by Yiqun Lin. 2019-05-16 17:04:46 +02:00
Márton Elek 9248b7d58b
HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. 2019-05-16 17:04:46 +02:00
Shashikant Banerjee e66ecc564a HDDS-1531. Disable the sync flag by default during chunk writes in Datanode (#820). Contributed by Shashikant Banerjee. 2019-05-16 18:38:52 +05:30
supratimdeka 9569015802 HDDS-1511. Space tracking for Open Containers in HDDS Volumes. Contributed by Supratim Deka (#812) 2019-05-15 10:26:01 -07:00
Hanisha Koneru 02c9efcb81
HDDS-1491. Ozone KeyInputStream seek() should not read the chunk file. (#795) 2019-05-13 20:49:52 -07:00
Vivek Ratnavel Subramanian b832e174b0 HDDS-1474. "ozone.scm.datanode.id" config should take path for a dir and not a file (#809) 2019-05-09 17:11:09 -07:00
Hanisha Koneru 2d31ccc1b3
Revert "HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792)" (#808) 
This reverts commit 18cca583ac.
 2019-05-09 11:18:46 -07:00
Vivek Ratnavel Subramanian 18cca583ac HDDS-1474. ozone.scm.datanode.id config should take path for a dir (#792) 2019-05-09 10:23:44 -07:00
avijayanhwx eb9c8900bc HDDS-1475 : Fix OzoneContainer start method. (#788) 2019-05-07 14:09:41 -07:00
Shashikant Banerjee 7f0e2c67e0 HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749). Contributed by Shashikant Banerjee 2019-05-08 01:00:10 +05:30
Shashikant Banerjee 01451a57de Revert "HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749)" 
This reverts commit d525633619.
 2019-05-08 00:49:15 +05:30
bshashikant d525633619
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749) 
HDDS-1395. Key write fails with BlockOutputStream has been closed exception (#749).
 2019-05-08 00:38:53 +05:30
avijayanhwx f194540520 HDDS-1448 : RatisPipelineProvider should only consider open pipeline while excluding dn for pipeline allocation. (#786) 2019-05-04 00:19:00 +05:30
Siddharth 1df679985b HDDS-1473. DataNode ID file should be human readable. (#781) 2019-05-02 13:59:14 -07:00
Márton Elek e2f0f72677
HDDS-1469. Generate default configuration fragments based on annotations 
Closes #773
 2019-05-02 12:14:43 +02:00
Vivek Ratnavel Subramanian 3cb1d09b2e
HDDS-1479. Update S3.md documentation 
Closes #791
 2019-05-02 11:49:37 +02:00
Márton Elek a2887f5c23
HDDS-1468. Inject configuration values to Java objects 
Closes #772
 2019-05-02 11:35:01 +02:00
Ajay Kumar 382a962d8d
Revert "HDDS-973. HDDS/Ozone fail to build on Windows." 
This reverts commit 5dae1a0c66.
 2019-04-29 19:46:20 -07:00
Elek, Márton 59816dff94 HDDS-1462. Fix content and format of Ozone documentation. Contributed by Elek, Marton. (#767) 2019-04-29 13:28:19 -07:00
Siddharth 5bca062d0e HDDS-1476. Fix logIfNeeded logic in EndPointStateMachine. (#779) 2019-04-29 12:05:38 -07:00
Xiaoyu Yao a96d16a843
HDDS-1455. Inconsistent naming convention with Ozone Kerberos configuration. Contributed by Xiaoyu Yao. (#762) 2019-04-29 11:18:11 -07:00
Ajay Yadav b434f558ca
HDDS-1471. Update ratis dependency to 0.3.0. Contributed by Ajay Kumar. (#777) 2019-04-28 20:57:59 -07:00
Bharat Viswanadham 43b2a4b77b
HDDS-1456. Stop the datanode, when any datanode statemachine state is… (#769) 2019-04-26 14:25:34 -07:00
Hanisha Koneru 37582705fa
HDDS-1403. KeyOutputStream writes fails after max retries while writing to a closed container (#753) 2019-04-26 10:39:03 -07:00
Eric Yang e95c055d91
HDDS-1457. Sequence of configuring Ozone File System instruction needs some work. Contributed by Eric Yang. 2019-04-24 15:52:51 +02:00
Kitti Nanasi 3f787cd506
HDDS-1192. Support -conf command line argument in GenericCli 
Closes #713
 2019-04-24 15:34:49 +02:00
Xiaoyu Yao 64f30da428 HDDS-1450. Fix nightly run failures after HDDS-976. Contributed by Xiaoyu Yao. (#757) 2019-04-23 21:38:23 -07:00
Siddharth 59ded7641f HDDS-1411. Add unit test to check if SCM correctly sends close commands for containers in closing state after a restart. (#755) 2019-04-23 21:04:14 +05:30
Nanda kumar 7e1f8d3a1b
HDDS-1368. Cleanup old ReplicationManager code from SCM. 2019-04-23 17:35:39 +05:30
Chen, Junjie 518f47bf9b HDDS-976: Parse network topology from yaml file. Contributed by Junjie Chen. (#661) 
Signed-off-by: Xiaoyu Yao <xyao@apache.org>
 2019-04-18 16:53:25 -07:00
Márton Elek 8aea5e4cb6
HDDS-1433. Rename GetScmInfoRespsonseProto to GetScmInfoResponseProto due to typos. Contributed by Wanqiang Ji. 2019-04-18 08:53:08 +02:00
Shashikant Banerjee d608be660f HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException.(#729) 2019-04-17 14:49:02 +05:30
Shashikant Banerjee 082f1e0437 Revert "HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729)" 
This reverts commit df2ae27f3e.
 2019-04-17 14:38:08 +05:30
Shashikant Banerjee df2ae27f3e HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException..(#729) 2019-04-17 14:25:35 +05:30
Shashikant Banerjee e5a918cda3 Revert "Merge pull request #729 from bshashikant/HDDS-1373" 
This reverts commit 4ff4314e40, reversing
changes made to 04c0437d13.
 2019-04-17 14:17:54 +05:30
bshashikant 4ff4314e40
Merge pull request #729 from bshashikant/HDDS-1373 
HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee.
 2019-04-17 14:12:52 +05:30
Doroszlai, Attila 0a87fd1464 HDDS-1434. TestDatanodeStateMachine is flaky (#740) 2019-04-16 19:13:01 -07:00
Hanisha Koneru e543c3b31a
HDDS-1376. Datanode exits while executing client command when scmId is null (#724) 2019-04-16 13:51:39 -07:00
Bharat Viswanadham 3ac3e50b20
HDDS-1374. ContainerStateMap cannot find container while allocating blocks. (#735) 2019-04-16 13:49:29 -07:00
Arpit Agarwal ebbda181e4
HDDS-1198. Rename chill mode to safe mode. Contributed by Siddharth Wagle. (#737) 2019-04-13 07:27:03 -07:00
Arpit Agarwal 62f4808617
HDDS-1428. Remove benign warning in handleCreateContainer. Contributed by Siddharth Wagle. 2019-04-12 15:08:01 -07:00
Bharat Viswanadham a458c5bd96
HDDS-1285. Implement actions need to be taken after chill mode exit wait time. (#612) 2019-04-12 09:40:27 -07:00
Arpit Agarwal abace709cc
HDDS-1420. Tracing exception in DataNode HddsDispatcher 
Closes #723
 2019-04-12 12:21:10 +02:00
Arpit Agarwal a9b46c58fe
HDDS-1421. Avoid unnecessary object allocations in TracingUtil 
Closes #722
 2019-04-12 11:52:31 +02:00
Doroszlai, Attila 3b08ac4659 HDDS-1363. ozone.metadata.dirs doesn't pick multiple dirs (#691) 2019-04-12 15:00:08 +05:30
Nanda kumar c298415eb5
HDDS-1404. Fix typos in HDDS. Contributed by bianqi. 2019-04-12 14:12:10 +05:30
Shashikant Banerjee 22d0468b39 HDDS-1373. KeyOutputStream, close after write request fails after retries, runs into IllegalArgumentException. Contributed by Shashikant Banerjee 2019-04-11 22:13:41 +05:30
Shashikant Banerjee a0468c5756 HDDS-1348. Refactor BlockOutpuStream Class. Contributed by Shashikant Banerjee. 2019-04-11 19:31:26 +05:30
Nanda kumar e9c4109004 HDDS-1417. After successfully importing a container, datanode should delete the container tar.gz file from working directory. (#721) 2019-04-10 16:55:37 -07:00
Bharat Viswanadham 0e770a6539
HDDS-1370. Command Execution in Datanode fails because of NPE (#715) 2019-04-10 10:25:28 -07:00
Márton Elek dfb518bbf5
HDDS-1397. Avoid the usage of signal handlers in datanodes of the MiniOzoneClusters. 
Closes #701
 2019-04-10 18:35:36 +02:00
Mukul Kumar Singh df01469141 HDDS-1401. Static ContainerCache in Datanodes can result in overwrite of container db. Contributed by Mukul Kumar Singh. (#708) 2019-04-10 18:00:10 +05:30
Mukul Kumar Singh 312d4d4cc5 HDDS-1388. Add a shell script to run MiniOzoneChaosCluster using mvn exec. Contributed by Mukul Kumar Singh. (#709) 2019-04-09 13:08:25 -07:00
Nanda kumar 73f43ac2dc
HDDS-1372. getContainerWithPipeline for a standalone pipeline fails with ConcurrentModificationException. (#682) 2019-04-09 15:10:07 +05:30
Shashikant Banerjee 2d4f6b6daa HDDS-372. There are three buffer copies in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-04-09 10:27:58 +05:30
Anu Engineer dc013f7a35 HDDS-1329. Update documentation for Ozone-0.4.0 release. 
Contributed By Ajay Kumar.
 2019-04-05 14:14:42 -07:00
Doroszlai, Attila 274ab4e86c HDDS-1332. Attempt to fix flaky test testStartStopDatanodeStateMachine (#697) 2019-04-05 13:21:58 -07:00
Nanda kumar 5750bb94ed HDDS-1389. Fix testSCMChillModeRestrictedOp. (#696) 2019-04-05 10:42:00 -07:00
Ajay Kumar 0356b3699d HDDS-1333. OzoneFileSystem can't work with spark/hadoop2.7 because incompatible security classes. Contributed by Elek, Marton. (#653) 
(cherry picked from commit 7ec6a31eb3)
 2019-04-05 09:29:21 -07:00
avijayanhwx 993f36ee33 HDDS-1353 : Metrics scm_pipeline_metrics_num_pipeline_creation_failed keeps increasing because of BackgroundPipelineCreator. (#681) 2019-04-04 19:18:32 +05:30
Shashikant Banerjee 33e159bf8d HDDS-1349. Remove watchClient from XceiverClientRatis. Contributed by Shashikant Banerjee. 2019-04-04 16:35:37 +05:30
Nanda kumar 48a58bce37
HDDS-1207. Refactor Container Report Processing logic and plugin new Replication Manager. (#662) 2019-04-04 16:32:59 +05:30
Yiqun Lin a92806d05a HDDS-1189. Recon Aggregate DB schema and ORM. Contributed by Siddharth Wagle. 2019-04-04 17:33:37 +08:00
Hanisha Koneru f09a78f73f
HDDS-1339. Implement ratis snapshots on OM (#651) 2019-04-03 22:50:28 -07:00
Yiqun Lin f96fb05a2b HDDS-1365. Fix error handling in KeyValueContainerCheck. Contributed by Supratim Deka. 2019-04-03 14:01:30 +08:00
Kitti Nanasi aeb43dfba3
HDDS-1153. Make tracing instrumentation configurable 
Closes #673
 2019-04-02 14:21:47 +02:00
Yiqun Lin d31c86892e HDDS-1337. Handle GroupMismatchException in OzoneClient. Contributed by Shashikant Banerjee. 2019-04-02 16:32:30 +08:00
Doroszlai, Attila 36756703f0 HDDS-1322. Hugo errors when building Ozone (#671) 2019-04-01 08:52:05 -07:00
Yiqun Lin 8bfef21efa HDDS-1360. Invalid metric type due to fully qualified class name. Contributed by Doroszlai, Attila. 2019-04-01 19:26:44 +08:00
Shweta Yakkali 82769ba1b8 HDDS-1352. Remove unused call in TestStorageContainerManagerHttpServer. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-29 17:26:48 +05:30
Xiaoyu Yao d7a2f9456b HDDS-1350. Fix checkstyle issue in TestDatanodeStateMachine. Contributed by Xiaoyu Yao. (#656) 2019-03-28 14:50:34 -07:00
nilotpalnandi 4cceeb2682 HDDS-1309 . change logging from warn to debug in XceiverClient. Contributed by Nilotpal Nandi. (#638) 2019-03-28 12:22:44 -07:00
Shashikant Banerjee ac4010bb22 HDDS-1293. ExcludeList#getProtoBuf throws ArrayIndexOutOfBoundsException. Contributed by Shashikant Banerjee. 2019-03-29 00:43:28 +05:30
Xiaoyu Yao ca5e4ce036
HDDS-1318. Fix MalformedTracerStateStringException on DN logs. Contributed by Xiaoyu Yao. 
This closes #641
 2019-03-28 12:00:58 -07:00
Arpit Agarwal b4ed81c4e6 HDDS-1332. Add some logging for flaky test testStartStopDatanodeStateMachine. Contributed by Arpit Agarwal. (#649) 2019-03-27 20:53:30 +05:30
Sandeep Nemuri 5c0a81ad3c HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-26 15:44:18 +05:30
Bharat Viswanadham 8739693514 HDDS-1217. Refactor ChillMode rules and chillmode manager. (#558) 2019-03-26 01:41:03 +05:30
Yiqun Lin e5d72f504e HDDS-1234. Iterate the OM DB snapshot and populate the recon container DB. Contributed by Aravindan Vijayan. 2019-03-25 22:52:02 +08:00
Shashikant Banerjee d4e4a7d456 HDDS-1317. KeyOutputStream#write throws ArrayIndexOutOfBoundsException when running RandomWrite MR examples. Contributed by Shashikant Banerjee. 2019-03-25 15:41:20 +05:30
Bharat Viswanadham 128dd91e10
HDDS-1331. In DatanodeStateMachine join check for not null. 2019-03-23 09:43:49 -07:00
Bharat Viswanadham 43e421afef
HDDS-918. Expose SCMMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-22 16:44:12 -07:00
Ajay Kumar 771ea6b5e7 Revert "HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri." 
This reverts commit efad5717ec.
 2019-03-22 16:20:44 -07:00
Ajay Kumar efad5717ec HDDS-1310. In datanode once a container becomes unhealthy, datanode restart fails. Contributed by Sandeep Nemuri. 2019-03-22 15:53:12 -07:00
Vivek Ratnavel Subramanian 73f7b04e2b HDDS-1302. Fix SCM CLI does not list container with id 1. 2019-03-22 15:15:01 -07:00
Arpit Agarwal f854a89190
HDDS-1205. Refactor ReplicationManager to handle QUASI_CLOSED containers. Contributed by Nanda kumar. (#620) 2019-03-22 14:36:29 -07:00
Arpit Agarwal 90afc9ab03
HDDS-1320. Update ozone to latest ratis snapshot build (0.4.0-1fc5ace-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-21 14:24:02 -07:00
Yiqun Lin 60cdd4cac1 HDDS-1233. Create an Ozone Manager Service provider for Recon. Contributed by Aravindan Vijayan. 2019-03-21 11:02:29 +08:00
Mukul Kumar Singh 072750c00a HDDS-1306. TestContainerStateManagerIntegration fails in Ratis shutdown. Contributed by Lokesh Jain. 2019-03-20 17:14:19 +05:30
Ajay Yadav f10d493325
HDDS-1119. DN get OM certificate from SCM CA for block token validation. Contributed by Ajay Kumar. (#601) 2019-03-18 23:08:17 -07:00
Arpit Agarwal ae3a2c3851
Revert "HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608)" 
This reverts commit 44b8451821.
 2019-03-18 14:21:57 -07:00
Sammi Chen 4d2a116d6e HDDS-699. Detect Ozone Network topology. Contributed by Sammi Chen. 2019-03-17 17:08:29 +08:00
Bharat Viswanadham 926d548caa
HDDS-1281. Fix the findbug issue caused by HDDS-1163. Contributed by Aravindan Vijayan. 2019-03-15 21:06:44 -07:00
Vivek Ratnavel Subramanian af2dfc9f3d HDDS-1263. SCM CLI does not list container with id 1. 2019-03-15 15:39:30 -07:00
Elek, Márton 44b8451821 HDDS-1284. Adjust default values of pipline recovery for more resilient service restart. Contributed by Elek, Marton. (#608) 2019-03-15 14:51:37 -07:00
Shashikant Banerjee 155ab6d5d8 HDDS-1098. Introduce Retry Policy in Ozone Client. Contributed by Shashikant Banerjee. 2019-03-15 15:40:59 +05:30
Vivek Ratnavel Subramanian 9001508485 HDDS-761. Create S3 subcommand to run S3 related operations. 
Closes #579
 2019-03-14 21:16:59 -07:00
Bharat Viswanadham 091a664977
HDDS-917. Expose NodeManagerMXBean as a MetricsSource. Contributed by Siddharth Wagle. 2019-03-14 12:41:52 -07:00
Shashikant Banerjee d60673c470 HDDS-1257. Incorrect object because of mismatch in block lengths. Contributed by Shashikant Banerjee. 2019-03-14 19:32:36 +05:30
Márton Elek 3d140a537a
HDDS-1247. Bump trunk ozone version to 0.5.0. Contributed by Elek, Marton. 
Closes #588
 2019-03-14 12:01:25 +01:00
Mukul Kumar Singh ea9fb07fd2 HDDS-1241. Update ozone to latest ratis snapshot build (0.4.0-5680cf5-SNAPSHOT). Contributed by Mukul Kumar Singh. 2019-03-14 15:18:06 +05:30
Ajay Yadav 4fa009989b
HDDS-1254. Fix failure in TestOzoneManagerHttpServer & TestStorageContainerManagerHttpServer. Contributed by Ajay Kumar. (#598) 2019-03-13 12:01:08 -07:00
Xiaoyu Yao 86d508c7c7 HDDS-1087. Fix TestDefaultCertificateClient#testSignDataStream. Contributed by Xiaoyu Yao. (#596) 2019-03-13 11:50:53 -07:00
Aravindan Vijayan 1f47fb7a2f HDDS-1209. Fix the block allocation logic in SCM when client wants to exclude all available open containers in a chosen pipeline. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 17:12:43 +05:30
Vivek Ratnavel Subramanian c1649bf1e1
HDDS-1226. Addendum. ozone-filesystem jar missing in hadoop classpath 
Closes #560
 2019-03-13 12:08:45 +01:00
Lokesh Jain 0d62753da9 HDDS-1128. Create stateful manager class for the pipeline creation scheduling. 
Signed-off-by: Nanda kumar <nanda@apache.org>
 2019-03-13 16:30:06 +05:30
Mukul Kumar Singh d602d462a6 HDDS-1256. Fix failure in TestKeyValueContainerCheck. Contributed by Supratim Deka. 2019-03-13 16:22:19 +05:30
Arpit Agarwal 24793d2d97
HDDS-1163. Basic framework for Ozone Data Scrubber. Contributed by Supratim Deka. 2019-03-13 04:32:39 +09:00
Márton Elek c449cdebe6
HDDS-807. Period should be an invalid character in bucket names. Contributed by Siddharth Wagle. 2019-03-12 19:14:55 +01:00
Ajay Yadav dcb0de848d
HDDS-1043. Enable token based authentication for S3 api 
Closes #561
 2019-03-12 18:47:06 +01:00
Xiaoyu Yao 2778259b88 HDDS-1253. Fix checkstyle issue from Nightly run. Contributed by Xiaoyu Yao. (#593) 
Closes PR #593
 2019-03-12 09:34:18 -07:00
Mukul Kumar Singh daf4660eeb HDDS-1095. OzoneManager#openKey should do multiple block allocations in a single SCM rpc call. Contributed by Mukul Kumar Singh. 2019-03-12 20:02:19 +05:30
Márton Elek d17e31e062
HDDS-1214. Enable tracing for the datanode read/write path. Contributed by Elek, Marton. 
Closes #550.
 2019-03-12 11:34:56 +01:00
Ajay Yadav 599e926d46 HDDS-596. Add robot test for OM Block Token. Contributed by Ajay Kumar. 
This closes (#581)
 2019-03-11 15:01:04 -07:00
Shashikant Banerjee b4aa24d3c5 HDDS-1173. Fix a data corruption bug in BlockOutputStream. Contributed by Shashikant Banerjee. 2019-03-11 23:15:49 +05:30
Bharat Viswanadham 68aa2ccd89
HDDS-1240. Fix check style issues caused by HDDS-1196. 2019-03-09 21:50:46 -08:00
Xiaoyu Yao 11db46956c
HDDS-1235. BaseHttpServer NPE is HTTP policy is HTTPS_ONLY. Contributed by Xiaoyu Yao. 
Closes #572
 2019-03-09 09:38:01 -08:00
Mukul Kumar Singh 8458ced11e HDDS-1238. Fix Ratis Snapshot creation error if the snapshot file already exists in ContainerStateMachine. Contributed by Shashikant Banerjee. 2019-03-09 22:46:20 +05:30
Bharat Viswanadham 341c076f05 HDDS-1196. Add a ReplicationStartTimer class. (#567) 2019-03-08 20:55:33 +05:30
Yiqun Lin fb851c9481 HDDS-1232. Recon Container DB service definition. Contributed by Aravindan Vijayan. 2019-03-08 16:59:41 +08:00
Ajay Yadav 064f38b3a5 HDDS-594. SCM CA: DN sends CSR and uses certificate issued by SCM. Contributed by Ajay Kumar. (#547) 2019-03-07 14:41:52 -08:00