Apache/hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,328 Commits 326 Branches 371 Tags
Commit Graph

329 Commits

Author SHA1 Message Date
Mukul Kumar Singh
feb43ecbb4 HDDS-824. WriteStateMachineData timesout leading to Datanode crash. Contributed by Mukul Kumar Singh. 2018-11-09 18:15:24 +05:30
Bharat Viswanadham
6bb401616c HDDS-812. TestEndPoint#testCheckVersionResponse is failing. Contributed by Hanisha Koneru. 
(cherry picked from commit 66bf62404a038193fe5849c84edaa062d8e05f44)
 2018-11-08 17:53:14 -08:00
Shashikant Banerjee
788cf061cd HDDS-806. Update Ratis to latest snapshot version in ozone. Contributed by Tsz Wo Nicholas Sze and Mukul Kumar Singh. 2018-11-09 00:59:48 +05:30
Akira Ajisaka
f2f3d47ab5
HDDS-811. [JDK 11] Fix compilation failures with jdk 11. Contributed by Lokesh Jain. 
(cherry picked from commit 49412a12898eda60cb460ebf4bb9126f3c8c1f53)
 2018-11-06 21:27:23 +09:00
Shashikant Banerjee
4b00044883 HDDS-799. Avoid ByteString to byte array conversion cost by using ByteBuffer in Datanode. Contributed by Mukul Kumar Singh. 2018-11-06 00:00:23 +05:30
Arpit Agarwal
53d4aefae8 HDDS-794. Add configs to set StateMachineData write timeout in ContainerStateMachine. Contributed by Shashikant Banerjee. 
(cherry picked from commit 408f59caa9321be8a55afe44b1811c5dacf23206)
 2018-11-05 10:10:17 -08:00
Arpit Agarwal
db90350c97 HDDS-797. If DN is started before SCM, it does not register. Contributed by Hanisha Koneru. 
(cherry picked from commit c8ca1747c08d905cdefaa5566dd58d770a6b71bd)
 2018-11-05 10:07:43 -08:00
Nanda kumar
73fbde31c6 HDDS-775. Batch updates to container db to minimize number of updates. 
Contributed by Mukul Kumar Singh.
 2018-11-01 17:56:34 +05:30
Yiqun Lin
73e9e43483 HDDS-786. Fix the findbugs for SCMClientProtocolServer#getContainerWithPipeline. 
(cherry picked from commit 2e8ac14dcb57a0fe07b2119c26535c3541665b70)
 2018-11-01 14:12:30 +08:00
Anu Engineer
382b73a109 HDDS-754. VolumeInfo#getScmUsed throws NPE. 
Contributed by Hanisha Koneru.

(cherry picked from commit 773f0d1519715e3ddf77c139998cc12d7447da66)
 2018-10-30 19:26:35 -07:00
Mukul Kumar Singh
5e48a70474 HDDS-721. NullPointerException thrown while trying to read a file when datanode restarted. Contributed by Shashikant Banerjee. 2018-10-30 19:45:32 +05:30
Arpit Agarwal
da32f6537b HDDS-620. ozone.scm.client.address should be an optional setting. Contributed by chencan and Arpit Agarwal. 
(cherry picked from commit 496f0ffe9017b11d0d7c071bad259d132687c656)
 2018-10-29 17:14:34 -07:00
Nanda kumar
38f96c5f72 HDDS-728. Datanodes should use different ContainerStateMachine for each pipeline. 
Contributed by Mukul Kumar Singh.
 2018-10-30 00:57:13 +05:30
Arpit Agarwal
9854eed1a4 HDDS-727. ozone.log is not getting created in logs directory. Contributed by Nilotpal Nandi. 
(cherry picked from commit a58048e8d545534e5b7f32e747df5a5f598889e2)
 2018-10-29 09:35:24 -07:00
Arpit Agarwal
2d8bb2ac02 HDDS-719. Remove Ozone dependencies on Apache Hadoop 3.2.0. Contributed by Arpit Agarwal. 
(cherry picked from commit 244afaba4a2dd7db830a0479941e11efb114cca0)
 2018-10-24 16:17:14 -07:00
Shashikant Banerjee
d37705c0f1 HDDS-716. Update ozone to latest ratis snapshot build(0.3.0-aa38160-SNAPSHOT). Contributed by Mukul Kumar Singh. 
(cherry picked from commit 0891cdda7961f7d0d7debdb8e89b7816f39f7c7b)
 2018-10-24 08:34:34 -07:00
Shashikant Banerjee
27bbbd0f9d Revert "HDDS-579. ContainerStateMachine should fail subsequent transactions per container in case one fails. Contributed by Shashikant Banerjee." 
This reverts commit a619d120a6c44bde2a846d61505a94f896e58e46.
This change introduces a new state for container replica called UNHEALTHY, which is not handled in SCM currently.
 2018-10-24 15:05:50 +05:30
Shashikant Banerjee
3191afa121 HDDS-676. Enable Read from open Containers via Standalone Protocol. Contributed by Shashikant Banerjee. 2018-10-23 19:19:58 +05:30
Shashikant Banerjee
575613ef7a Revert "HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee." 
This reverts commit a492edee4b84e38552f185a4227b49abf60f762d.
The commit msg is wrong.
 2018-10-23 19:14:18 +05:30
Shashikant Banerjee
a492edee4b HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee. 2018-10-23 19:02:12 +05:30
Nanda kumar
346afb0a5c HDDS-695. Introduce new SCM Commands to list and close Pipelines. 
Contributed by Nanda kumar.
 2018-10-23 18:41:30 +05:30
Shashikant Banerjee
0674f11fc0 HDDS-708. Validate BCSID while reading blocks from containers in datanodes. Contributed by Shashikant Banerjee. 
(cherry picked from commit b61846392ee589a0217e95defbfa824f29088057)
 2018-10-23 17:01:24 +05:30
Mukul Kumar Singh
e26f26c4c2 HDDS-638. Enable ratis snapshots for HDDS datanodes. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 82919a1e7af2ec22993f273e9ee64512a688c99b)
 2018-10-22 21:56:09 +05:30
Anu Engineer
34650b18e4 HDDS-544. Unconditional wait findbug warning from ReplicationSupervisor. 
Contributed by Arpit Agarwal.

(cherry picked from commit 7db3bb3ac185929fbe8f5b9b59cdd157eec2ac64)
 2018-10-21 23:20:32 -07:00
Anu Engineer
2928f952f0 HDDS-707. Allow registering MBeans without additional jmx properties. 
Contributed by Arpit Agarwal.

(cherry picked from commit c9077a9f5a693df63c2ff797415605233e38a9a8)
 2018-10-21 23:00:09 -07:00
Nandakumar
eeb1d432a8 HDDS-687. Exception while loading containers during SCM startup. Contributed by Lokesh Jain. 2018-10-18 18:03:33 +05:30
Jitendra Pandey
106be1416d HDDS-651. Rename o3 to o3fs for Filesystem. 2018-10-17 14:32:22 -07:00
Nandakumar
51a79cc71d HDDS-661. When a volume fails in datanode, VersionEndpointTask#call ends up in dead lock. Contributed by Hanisha Koneru. 
(cherry picked from commit d54f5598f4ccd1031e8295a215a3183f3647031a)
 2018-10-17 19:00:19 +05:30
Márton Elek
b2ab592e0a HDDS-527. Show SCM chill mode status in SCM UI. Contributed by Yiqun Lin. 
(cherry picked from commit a9a63ae4a8367e66d5ec86b0097326b8491e4b1e)
 2018-10-17 13:01:27 +02:00
Arpit Agarwal
1f55beb527 HDDS-378. Remove dependencies between hdds/ozone and hdfs proto files. Contributed by Elek, Marton. 
(cherry picked from commit 5f0b43fa93d02c90956d48eb5c6c0b60deeac91c)
 2018-10-16 10:40:07 -07:00
Jitendra Pandey
e4dfdcee9f HDDS-667. Fix TestOzoneFileInterfaces. Contributed by Mukul Kumar Singh. 2018-10-16 10:36:37 -07:00
Arpit Agarwal
15d4e0110c HDDS-490. Improve om and scm start up options . Contributed by Namit Maheshwari. 
(cherry picked from commit 2614078b213e03085635b4455ed391438eef8db2)
 2018-10-15 16:45:16 -07:00
Jitendra Pandey
b51bc6b93f HDDS-629. Make ApplyTransaction calls in ContainerStateMachine idempotent. Contributed by Shashikant Banerjee. 2018-10-15 13:31:08 -07:00
Jitendra Pandey
a619d120a6 HDDS-579. ContainerStateMachine should fail subsequent transactions per container in case one fails. Contributed by Shashikant Banerjee. 2018-10-15 13:30:53 -07:00
Jitendra Pandey
0008357940 HDDS-603. Add BlockCommitSequenceId field per Container and expose it in Container Reports. Contributed by Shashikant Banerjee. 2018-10-15 13:30:39 -07:00
Anu Engineer
39ffed2a11 HDDS-653. TestMetadataStore#testIterator fails on Windows. 
Contributed by Yiqun Lin.

(cherry picked from commit 9227f3d22412f94d22f4cae6a4fb3a52ead9b011)
 2018-10-13 10:46:58 -07:00
Nandakumar
c141a912f0 HDDS-587. Add new classes for pipeline management. Contributed by Lokesh Jain. 
(cherry picked from commit 5c8e023ba32da3e65193f6ced354efe830dba75d)
 2018-10-13 03:23:53 +05:30
Arpit Agarwal
86c9418c16 HDDS-644. Rename dfs.container.ratis.num.container.op.threads. Contributed by Bharat Viswanadham. 
(cherry picked from commit 3a684a2b23517df996a8c30731d9b5cb3282cc2a)
 2018-10-12 14:44:56 -07:00
Anu Engineer
7daeaa9a98 HDDS-624.PutBlock fails with Unexpected Storage Container Exception. 
Contributed by  Arpit Agarwal.

(cherry picked from commit 02e1ef5e0779369f1363df2bb0437945fe9a271c)
 2018-10-12 14:29:59 -07:00
Xiaoyu Yao
4a0e137e04 HDDS-555. RandomKeyGenerator runs not closing the XceiverClient properly. Contributed by Mukul Kumar Singh. 
(cherry picked from commit e12edb3d8f05ff6206c6de1b81ecfa1a03abb193)
 2018-10-12 13:26:29 -07:00
Bharat Viswanadham
67d516eb8a HDDS-609. On restart, SCM does not exit chill mode as it expects DNs to report containers in ALLOCATED state. Contributed by Hanisha Koneru. 
(cherry picked from commit 74db993a61e201d2e8d33bb42f787e0a71388d0d)
 2018-10-11 21:04:19 -07:00
Tsz Wo Nicholas Sze
d72acd6bd3 HDDS-625. putKey hangs for a long time after completion, sometimes forever. Contributed by Arpit Agarwal 2018-10-12 10:49:33 +08:00
Arpit Agarwal
ed09883f2c HDDS-636. Turn off ozone.metastore.rocksdb.statistics for now. Contributed by Arpit Agarwal. 
(cherry picked from commit 50552479c42d2e84cfeec83d4d67b050c5659b06)
 2018-10-11 17:21:46 -07:00
Jitendra Pandey
077d36d8f5 HDDS-550. Serialize ApplyTransaction calls per Container in ContainerStateMachine. Contributed by Shashikant Banerjee. 2018-10-11 17:15:08 -07:00
Hanisha Koneru
43ff801391 HDDS-601. On restart, SCM throws 'No such datanode' exception. 
(cherry picked from commit eb34b5f8aff538e90c7820c0114322d2dd75f6e6)
 2018-10-11 13:48:02 -07:00
Arpit Agarwal
d7d445b354 HDDS-619. hdds.db.profile should not be tagged as a required setting & should default to DISK. Contributed by Dinesh Chitlangia. 
(cherry picked from commit 2bd000c85195416b9bcd06a3abb5100aeeda9727)
 2018-10-10 17:37:29 -07:00
Nanda kumar
79d11e50a7 HDDS-608. Access denied exception for user hdfs while making getContainerWithPipeline call. Contributed by Mukul Kumar Singh. 
(cherry picked from commit 90552b1ea8380c1155f7000bd9e4c23e709e6970)
 2018-10-10 20:51:23 +05:30
Bharat Viswanadham
056f480093 HDDS-583. SCM returns zero as the return code, even when invalid options are passed. Contributed by Namit Maheshwari. 
(cherry picked from commit 6a39739316795a4828833e99d78aadc684270f98)
 2018-10-09 15:21:50 -07:00
Nanda kumar
abfd4ffdc9 HDDS-450. Generate BlockCommitSequenceId in ContainerStateMachine for every commit operation in Ratis. Contributed by Shashikant Banerjee. 
(cherry picked from commit 7367ff333bf332b300e0acd6e7501ce8139a1998)
 2018-10-09 22:31:19 +05:30
Nanda kumar
4ba6dd4605 HDDS-599. Fix TestOzoneConfiguration TestOzoneConfigurationFields. Contributed by Sandeep Nemuri. 
(cherry picked from commit 9e9915ddab7ff79224b075889f67c38920fde9f0)
 2018-10-09 22:30:31 +05:30