mirror of https://github.com/apache/druid.git
39 lines
1.0 KiB
Markdown
39 lines
1.0 KiB
Markdown
|
---
|
||
|
|
layout: doc_page
|
||
|
|
---
|
||
|
|
|
||
|
|
# S3-compatible
|
||
|
|
|
||
|
|
Make sure to [include](../../operations/including-extensions.html) `druid-s3-extensions` as an extension.
|
||
|
|
|
||
|
|
## Deep Storage
|
||
|
|
|
||
|
|
S3-compatible deep storage is basically either S3 or something like Google Storage which exposes the same API as S3.
|
||
|
|
|
||
|
|
### Configuration
|
||
|
|
|
||
|
|
|Property|Possible Values|Description|Default|
|
||
|
|
|--------|---------------|-----------|-------|
|
||
|
|
|`druid.s3.accessKey`||S3 access key.|Must be set.|
|
||
|
|
|`druid.s3.secretKey`||S3 secret key.|Must be set.|
|
||
|
|
|`druid.storage.bucket`||Bucket to store in.|Must be set.|
|
||
|
|
|`druid.storage.baseKey`||Base key prefix to use, i.e. what directory.|Must be set.|
|
||
|
|
|
||
|
|
## StaticS3Firehose
|
||
|
|
|
||
|
|
This firehose ingests events from a predefined list of S3 objects.
|
||
|
|
|
||
|
|
Sample spec:
|
||
|
|
|
||
|
|
```json
|
||
|
|
"firehose" : {
|
||
|
|
"type" : "static-s3",
|
||
|
|
"uris": ["s3://foo/bar/file.gz", "s3://bar/foo/file2.gz"]
|
||
|
|
}
|
||
|
|
```
|
||
|
|
|
||
|
|
|property|description|default|required?|
|
||
|
|
|--------|-----------|-------|---------|
|
||
|
|
|type|This should be "static-s3"|N/A|yes|
|
||
|
|
|uris|JSON array of URIs where s3 files to be ingested are located.|N/A|yes|
|