c7d426daf0
HDDS-1881. Design doc: decommissioning in Ozone ( #1196 )
...
* Address comments from Anu
* In-flight calculation is removed from the base model
* Throttling is mentioned but will be discussed in a separated document
* Model is improved to handle the over-replication in a specific way
(to handle Nanda's use-case)
* Stop conditions are defined independently from the model
* Style of the table fragments are improved
* typo fix (thx to sodonnell)
* remove white spaces from the end of the lines
* example is restored
* minor whitespace improvement to support pdf conversion
2019-08-28 10:19:03 -07:00
1407414a52
HDDS-1094. Performance test infrastructure : skip writing user data on Datanode. Contributed by Supratim Deka ( #1323 )
2019-08-28 10:05:20 -07:00
dc72782008
HDDS-738. Removing REST protocol support from OzoneClient.
...
Contributed by Elek, Marton.
2019-08-28 09:53:31 -07:00
c0499bd704
HDDS-1596. Create service endpoint to download configuration from SCM.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-28 09:13:46 -07:00
5d31a4eff7
HDDS-1753. Datanode unable to find chunk while replication data using ratis. ( #1318 )
2019-08-28 10:54:44 +05:30
b1eee8b52e
HDDS-1946. CertificateClient should not persist keys/certs to ozone.m… ( #1311 )
2019-08-27 17:29:27 -07:00
2b9cc7eb95
HDDS-2037. Fix hadoop version in pom.ozone.xml.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-27 16:14:01 -07:00
0883ce1021
HDDS-2026. Overlapping chunk region cannot be read concurrently
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-27 15:34:58 -07:00
66cfa482c4
HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee( #1226 ).
2019-08-27 23:38:43 +05:30
ce8eb1283a
Revert "HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee."
...
This reverts commit 62445021d5
2019-08-27 23:23:44 +05:30
4379370fb1
HDDS-1981: Datanode should sync db when container is moved to CLOSED or QUASI_CLOSED state ( #1319 )
2019-08-27 10:22:03 +05:30
b661dcf563
HDDS-2002. Update documentation for 0.4.1 release.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-23 19:04:13 -07:00
d3fe993e60
HDDS-2023. Fix rat check failures in trunk (addendum)
2019-08-23 12:31:42 +02:00
e2a55482ee
HDDS-2023. Fix rat check failures in trunk
...
Closes #1342
2019-08-23 12:02:45 +02:00
b4a95a2b00
HDDS-2000. Don't depend on bootstrap/jquery versions from hadoop-trunk snapshot
...
Closes #1339
2019-08-23 10:56:00 +02:00
4028cac56d
HDDS-1347. In OM HA getS3Secret call Should happen only leader OM. ( #670 )
2019-08-22 15:00:17 -07:00
28fb4b527a
HDDS-2008 : Wrong package for RatisHelper class in hadoop-hdds/common module. ( #1333 )
2019-08-22 10:57:22 -07:00
2ae7f444bd
HDDS-1999. Basic acceptance test and SCM/OM web UI broken by Bootstrap upgrade ( #1327 )
2019-08-21 12:16:14 -07:00
62445021d5
HDDS-1610. applyTransaction failure should not be lost on restart. Contributed by Shashikant Banerjee.
2019-08-20 14:37:01 +05:30
d69a1a0aa4
HDDS-1105 : Add mechanism in Recon to obtain DB snapshot 'delta' updates from Ozone Manager ( #1259 )
2019-08-19 10:47:49 -07:00
12c7084be3
HDDS-1971. Update document for HDDS-1891: Ozone fs shell command should work with default port when port number is not specified ( #1306 )
2019-08-18 18:29:52 -07:00
e32f52c75f
HDDS-1903 : Use dynamic ports for SCM in TestSCMClientProtocolServer … ( #1303 )
2019-08-18 10:08:38 -07:00
e61825682a
HDDS-1979. Fix checkstyle errors ( #1312 )
2019-08-17 21:53:12 -07:00
02015e3ed8
HDDS-1959. Decrement purge interval for Ratis logs in datanode ( #1301 )
2019-08-17 16:54:33 +05:30
bf3751521b
HDDS-1894. Add filter to scmcli listPipelines. ( #1286 )
2019-08-16 14:33:04 +08:00
77d102ceba
HDDS-1910. Cannot build hadoop-hdds-config from scratch in IDEA
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-15 11:03:41 -07:00
c801f7a26c
HDDS-1768. Audit xxxAcl methods in OzoneManager ( #1204 )
2019-08-15 08:43:47 -07:00
fc229b6490
HDDS-1832 : Improve logging for PipelineActions handling in SCM and datanode. (Change to Error logging)
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-13 22:04:31 -07:00
bf457797f6
HDDS-1920. Place ozone.om.address config key default value in ozone-site.xml
...
Change-Id: Ic5970b383357147b74a01680aedf40bed4d3e176
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-13 20:27:37 -07:00
50a22b66c0
HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject ( #950 )
...
* HDDS-1659. Define the process to add proposal/design docs to the Ozone subproject
* Remove Site improvements to display proposals
* adding license header
* clarify the support of the existing method
2019-08-13 17:10:36 -07:00
69b74e9016
HDDS-1488. Scm cli command to start/stop replication manager.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-13 14:57:05 -07:00
689a80d3ce
HDDS-1886. Use ArrayList#clear to address audit failure scenario
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-08-13 14:37:32 -07:00
14a4ce3cee
HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. Contributed by Siyao Meng.
2019-08-08 13:38:10 -07:00
397a5633af
HDDS-1888. Add containers to node2container map in SCM as part of ICR processing.
...
Signed-off-by: Nanda kumar <nanda@apache.org>
2019-08-08 15:22:03 +05:30
70b4617cfe
HDDS-1900. Remove UpdateBucket handler which supports add/remove Acl. ( #1219 )
2019-08-07 17:07:40 -07:00
8f9245bc2d
HDDS-1865. Use "ozone.network.topology.aware.read" to control both RPC client and server side logic ( #1184 )
2019-08-07 16:59:13 -07:00
0520f5cede
HDDS-1924. ozone sh bucket path command does not exist
...
Closes #1245
2019-08-07 18:21:37 +02:00
a63023f261
HDDS-1901. Fix Ozone HTTP WebConsole Authentication. Contributed by Xiaoyu Yao. ( #1228 )
2019-08-06 12:07:53 -07:00
2662e20580
HDDS-1905. PipelineActionHandler is not closing the pipeline when close action is received. ( #1227 )
2019-08-06 14:36:04 +05:30
f8ea6e1ce1
HDDS-1798. Propagate failure in writeStateMachineData to Ratis. Contributed by Supratim Deka ( #1113 )
2019-08-05 13:43:41 +05:30
065cbc6b54
HDDS-1896. Suppress WARN log from NetworkTopology#getDistanceCost. Contributed by Xiaoyu Yao. ( #1214 )
2019-08-03 22:33:01 -07:00
f4df97fd89
HDDS-1870. ConcurrentModification at PrometheusMetricsSink ( #1179 )
2019-08-03 22:26:25 -07:00
ec1d453846
HDDS-1788. Fix kerberos principal error in Ozone Recon. ( #1201 )
2019-08-03 10:49:07 -07:00
6e8c8413ee
HDDS-1832 : Improve logging for PipelineActions handling in SCM and datanode. ( #1200 )
2019-08-02 22:31:00 +05:30
946e301731
HDDS-1878. checkstyle error in ContainerStateMachine ( #1195 )
2019-08-02 22:27:54 +05:30
e111789aeb
HDDS-1882. TestReplicationManager failed with NPE. ( #1197 )
2019-08-02 01:03:57 +05:30
8bda91d20a
HADOOP-16398. Exports Hadoop metrics to Prometheus ( #1170 )
2019-07-31 10:11:36 -07:00
d4ab9aea6f
HDDS-1816: ContainerStateMachine should limit number of pending apply transactions. Adds a config, uses snapshot threshold default value. ( #1150 )
2019-07-31 13:26:24 +05:30
68f9f2d833
Revert HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated. ( #1183 )
2019-07-29 16:44:48 -07:00
4d07134b13
HDDS-1829 On OM reload/restart OmMetrics#numKeys should be updated ( #1164 )
2019-07-29 16:42:00 -07:00
60325c9611
HDDS-1391 : Add ability in OM to serve delta updates through an API. ( #1033 )
2019-07-29 09:39:40 -07:00
d023663e3e
HDDS-1833. Moved RefCountedDB stacktrace to log level trace.
...
Contributed by Siddharth Wagle
2019-07-29 12:05:24 -04:00
b039f7591f
HDDS-1682. TestEventWatcher.testMetrics is flaky
...
Closes #962 .
2019-07-29 11:04:45 +02:00
902ff4a2f6
HDDS-1867. Invalid Prometheus metric name from JvmMetrics
...
Closes #1172
2019-07-29 10:27:55 +02:00
c7c7a889a8
HDDS-1839: Change topology sorting related logs in Pipeline from INFO to DEBUG. Contributed by Junjie Chen.
...
This closes #1158 .
2019-07-26 08:14:24 -07:00
3426777140
HDDS-1861. Fix TableCacheImpl cleanup logic. ( #1165 )
2019-07-25 22:30:06 -07:00
2b1d8aedbb
HDDS-1850. ReplicationManager should consider inflight replication and deletion while picking datanode for re-replication.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-25 14:25:39 -07:00
b41ef61ebc
HDDS-1858. mTLS support for Ozone is not correct. Contributed by Siddharth Wagle.
2019-07-25 09:52:02 -07:00
ba43233451
HDDS-1749 : Ozone Client should randomize the list of nodes in pipeline for reads. ( #1124 )
2019-07-25 20:24:46 +05:30
2546e6ece2
HDDS-1817. GetKey fails with IllegalArgumentException.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-24 15:59:30 -07:00
fbe2fc6580
HDDS-1846. Default value for checksum bytes is different in ozone-site.xml and code.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-23 10:42:16 -07:00
278390f5bf
HDDS-1710. Fix checkstyle errors
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-23 07:56:30 -07:00
c533b79c32
HDDS-1710. Publish JVM metrics via Hadoop metrics
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 16:27:50 -07:00
c958eddcf4
HDDS-1811. Prometheus metrics are broken.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-22 14:12:06 -07:00
cdc36fe286
HDDS-1649. On installSnapshot notification from OM leader, download checkpoint and reload OM state ( #948 )
2019-07-22 12:06:55 -07:00
69a46a95bb
HDDS-1713. ReplicationManager fail to find proper node topology based… ( #1112 )
2019-07-19 15:45:26 -07:00
4e66cb9333
HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao. ( #1067 )
...
* HDDS-1653. Add option to "ozone scmcli printTopology" to order the output acccording to topology layer. Contributed by Xiaoyu Yao.
* use ip/hostname instead of network name for -o output and add smoke test
2019-07-19 11:00:49 +08:00
d5ef38b093
HDDS-1822. NPE in SCMCommonPolicy.chooseDatanodes ( #1120 )
2019-07-18 13:28:03 -07:00
20cf50c6d0
HDDS-1654. Ensure container state on datanode gets synced to disk whennever state change happens. Cotributed by Shashikant Banerjee. ( #923 )
2019-07-18 17:09:05 +05:30
ccceedb432
HDDS-1780. TestFailureHandlingByClient tests are flaky. Contributed by Shashikant Banerjee. ( #1073 )
2019-07-18 16:01:58 +05:30
23e9bebe13
HDDS-1767: ContainerStateMachine should have its own executors for executing applyTransaction calls ( #1087 )
2019-07-18 14:48:13 +05:30
3dc256ef81
HDDS-1689. Implement S3 Create Bucket request to use Cache and DoubleBuffer. ( #1088 )
2019-07-17 21:46:27 -07:00
73e6ffce69
HDDS-1802. Add Eviction policy for table cache. ( #1100 )
2019-07-17 18:19:06 -07:00
303a7f8a39
HDDS-1721 : Client Metrics are not being pushed to the configured sink while running a hadoop command to write to Ozone. ( #1034 )
2019-07-17 12:05:43 -07:00
eb0849b547
HDDS-1801. Make Topology Aware Replication/Read non-default for ozone 0.4.1. Contributed by Xiaoyu Yao.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-07-17 10:32:59 -07:00
493b0b5760
HDDS-1787. NPE thrown while trying to find DN closest to client. Contributed by Sammi Chen. ( #1094 )
2019-07-16 12:52:29 -07:00
d21eccf8ba
HDDS-1492. Generated chunk size name too long. Contributed by Shashikannt Banerjee. ( #1084 )
2019-07-16 18:01:13 +05:30
1502e6a22f
HDDS-1756. DeleteContainerCommandHandler fails with NPE. Contributed by Nanda kumar( #1095 ).
2019-07-16 15:06:41 +05:30
47345e598e
Merge pull request #1058 from elek/HDDS-1763
...
HDDS-1763. Use vendor neutral s3 logo in ozone doc
2019-07-15 13:17:21 -07:00
0976f6fc30
HDDS-1766. ContainerStateMachine is unable to increment lastAppliedTermIndex. Contributed by Mukul Kumar Singh. ( #1072 )
2019-07-14 10:53:51 +05:30
9119ed07ff
HDDS-1384. TestBlockOutputStreamWithFailures is failing
...
Closes #1029
2019-07-12 11:00:03 +02:00
14c43f85de
HDDS-1752. ConcurrentModificationException while handling DeadNodeHandler event. ( #1080 )
2019-07-12 14:15:48 +05:30
738fab3bff
HDDS-1754. getContainerWithPipeline fails with PipelineNotFoundException. Contributed by Supratim Deka ( #1081 )
2019-07-12 10:31:48 +05:30
0f399b0d57
HDDS-1784. Missing HostName and IpAddress in the response of register command.
2019-07-11 19:01:06 +05:30
acef5e0cec
HDDS-1201. Reporting corrupted containers info to SCM ( #1032 )
2019-07-11 14:49:58 +05:30
cdb20adfcc
HDDS-1611. Evaluate ACL on volume bucket key and prefix to authorize access. Contributed by Ajay Kumar. ( #973 )
2019-07-10 11:03:58 -07:00
e0fa1040af
HDDS-1748. Error message for 3 way commit failure is not verbose. Contributed by Supratim Deka ( #1051 )
2019-07-10 16:45:55 +05:30
ac7a8accdf
HDDS-1603. Handle Ratis Append Failure in Container State Machine. Contributed by Supratim Deka ( #1019 )
2019-07-10 07:49:36 +05:30
030307226a
HDDS-1586. Allow Ozone RPC client to read with topology awareness. Contributed by Sammi Chen.
2019-07-09 14:43:55 -07:00
96d0555913
HDDS-1718. Increase Ratis Leader election timeout default. Contributed by Aravindan Vijayan & Siddharth Wagle. ( #1062 )
2019-07-09 23:17:50 +05:30
82d88a8d30
HDDS-1705. Recon: Add estimatedTotalCount to the response of containers and containers/{id} endpoints. Contributed by Vivek Ratnavel Subramanian.
2019-07-08 21:06:50 -07:00
e5ffb88257
HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh. ( #1050 )
...
* HDDS-1550. MiniOzoneCluster is not shutting down all the threads during shutdown. Contributed by Mukul Kumar Singh.
2019-07-08 20:24:12 -07:00
b5d30e4914
HDDS-1750. Add block allocation metrics for pipelines in SCM. Contributed by Lokesh Jain. ( #1047 )
2019-07-09 08:42:55 +05:30
ce91d35b29
HDDS-1728. Add metrics for leader's latency in ContainerStateMachine. Contributed by Mukul Kumar Singh. ( #1022 )
2019-07-08 12:19:16 +05:30
68f53f95ad
HDDS-1763. Use vendor neutral s3 logo in ozone doc. Contributed by Elek, Marton.
2019-07-04 17:14:19 +02:00
8b0d1adf31
HDDS-1616. ManagedChannel references are being leaked in while removing RaftGroup. Contributed by Mukul Kumar Singh. ( #1039 )
2019-07-02 14:34:49 -07:00
f8d62a9c4c
HDDS-1258. Fix error propagation for SCM protocol
...
Closes #1001
2019-07-01 20:04:32 +02:00
1f75660350
HDDS-373. Genconf tool must generate ozone-site.xml with sample values ( #1025 )
2019-07-01 08:58:47 -07:00
9df6275954
HDDS-1555. Disable install snapshot for ContainerStateMachine. Contributed by Siddharth Wagle. ( #846 )
2019-07-01 20:52:32 +05:30
da568996af
HDDS-1733. Fix Ozone documentation ( #1031 )
2019-06-28 15:59:33 -07:00
Elek, Márton
supratimdeka
Anu Engineer
Márton Elek
bshashikant
Vivek Ratnavel Subramanian
Nanda kumar
Doroszlai, Attila
Shashikant Banerjee
Lokesh Jain
Bharat Viswanadham
avijayanhwx
Doroszlai, Attila
Siyao Meng
HUAN-PING SU
Li Cheng
dineshchitlangia
Aravindan Vijayan
Siyao Meng
Sammi Chen
Xiaoyu Yao
Akira Ajisaka
Eric Yang
Chen, Junjie
Hanisha Koneru
Mukul Kumar Singh
Hrishikesh Gadre
Ajay Yadav
S O'Donnell