HDDS-677. Create documentation for s3 gateway to the docs. Contributed by Elek Marton.

2018-10-31 11:48:23 -07:00 · 2018-10-31 11:48:23 -07:00 · 6668c19daf
parent 08bb0362e0
commit 6668c19daf
1 changed files with 130 additions and 0 deletions
--- a/hadoop-ozone/docs/content/S3.md
+++ b/hadoop-ozone/docs/content/S3.md
@ -0,0 +1,130 @@
+---
+title: S3
+menu:
+   main:
+      parent: Client
+      weight: 1
+---
+
+<!---
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with
+  this work for additional information regarding copyright ownership.
+  The ASF licenses this file to You under the Apache License, Version 2.0
+  (the "License"); you may not use this file except in compliance with
+  the License.  You may obtain a copy of the License at
+
+      http://www.apache.org/licenses/LICENSE-2.0
+
+  Unless required by applicable law or agreed to in writing, software
+  distributed under the License is distributed on an "AS IS" BASIS,
+  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+  See the License for the specific language governing permissions and
+  limitations under the License.
+-->
+
+Ozone provides S3 compatible REST interface to use the object store data with any S3 compatible tools.
+
+## Getting started
+
+S3 Gateway is a separated component which provides the S3 compatible. It should be started additional to the regular Ozone components.
+
+You can start a docker based cluster, including the S3 gateway from the release package.
+
+Go to the `compose/ozones3` directory, and start the server:
+
+```bash
+docker-compose up -d
+```
+
+You can access the S3 gateway at `http://localhost:9878`
+
+## URL Schema
+
+Ozone S3 gateway supports both the virtual-host-style URL s3 bucket addresses (eg. http://bucketname.host:9878) and the path-style addresses (eg. http://host:9878/bucketname)
+
+By default it uses the path-style addressing. To use virtual host style URLs set your main domain name in your `ozone-site.xml`:
+
+```xml
+<property>
+   <name>ozone.s3g.domain.name</name>
+   <value>s3g.internal</value>
+</property>
+```
+
+## Bucket browser
+
+Bucket's could be browsed from the browser with adding `?browser=true` to the bucket URL.
+
+For example the content of the 'testbucket' could be checked from the browser using the URL http://localhost:9878/testbucket?browser=true
+
+
+## Implemented REST endpoints
+
+Operations on S3Gateway service:
+
+Endpoint    | Status      |
+------------|-------------|
+GET service | implemented |
+
+Operations on Bucket:
+
+Endpoint                            | Status      | Notes
+------------------------------------|-------------|---------------
+GET Bucket (List Objects) Version 2 | implemented |
+HEAD Bucket                         | implemented |
+DELETE Bucket                       | implemented |
+PUT Bucket (Create bucket)          | implemented |
+Delete Multiple Objects (POST)      | implemented |
+
+Operation on Objects:
+
+Endpoint                            | Status          | Notes
+------------------------------------|-----------------|---------------
+PUT Object                          | implemented     |
+GET Object                          | implemented     | Range headers are not supported
+Multipart Uplad                     | not implemented |
+DELETE Object                       | implemented     |
+HEAD Object                         | implemented     |
+
+
+## Security
+
+Security is not yet implemented, you can *use* any AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
+
+Note: Ozone has a notion for 'volumes' which is missing from the S3 Rest endpoint. Under the hood S3 bucket names are maped to Ozone 'volume/bucket' locations (depending from the given authentication information).
+
+To show the storage location of a S3 bucket, use the `ozone sh bucket path <bucketname>` command.
+
+```
+aws s3api --endpoint-url http://localhost:9878 create-bucket --bucket=bucket1
+
+ozone sh bucket path bucket1
+Volume name for S3Bucket is : s3thisisakey
+Ozone FileSystem Uri is : o3fs://bucket1.s3thisisakey
+```
+
+## Clients
+
+### AWS Cli
+
+`aws` CLI could be used with specifying the custom REST endpoint.
+
+```
+aws s3api --endpoint http://localhost:9878 create-bucket --bucket buckettest
+```
+
+Or
+
+```
+aws s3 ls --endpoint http://localhost:9878 s3://buckettest
+```
+
+### S3 Fuse driver (goofys)
+
+Goofys is a S3 FUSE driver. It could be used to any mount any Ozone bucket as posix file system:
+
+
+```
+goofys --endpoint http://localhost:9878 bucket1 /mount/bucket1
+```