Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Doc: List supported values for Kafka `headerFormat` (#14316)

This commit is contained in:
Victoria Lim 2023-05-22 15:41:07 -07:00 committed by GitHub
parent 3f6610aaf1
commit 6b3a6113c4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 10 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
docs/development/extensions-core/kafka-ingestion.md
View File

@ -153,7 +153,13 @@ You would configure it as follows:
- `valueFormat`: Define how to parse the payload value. Set this to the payload parsing input format (`{ "type": "json" }`).
- `timestampColumnName`: Supply a custom name for the Kafka timestamp in the Druid schema to avoid conflicts with columns from the payload. The default is `kafka.timestamp`.
- `headerFormat`: The default "string" decodes UTF8-encoded strings from the Kafka header. If you need another format, you can implement your own parser.
- `headerFormat`: The default value `string` decodes strings in UTF-8 encoding from the Kafka header.
Other supported encoding formats include the following:
- `ISO-8859-1`: ISO Latin Alphabet No. 1, that is, ISO-LATIN-1.
- `US-ASCII`: Seven-bit ASCII. Also known as ISO646-US. The Basic Latin block of the Unicode character set.
- `UTF-16`: Sixteen-bit UCS Transformation Format, byte order identified by an optional byte-order mark.
- `UTF-16BE`: Sixteen-bit UCS Transformation Format, big-endian byte order.
- `UTF-16LE`: Sixteen-bit UCS Transformation Format, little-endian byte order.
- `headerColumnPrefix`: Supply a prefix to the Kafka headers to avoid any conflicts with columns from the payload. The default is `kafka.header.`.
Considering the header from the example, Druid maps the headers to the following columns: `kafka.header.env`, `kafka.header.zone`.
- `keyFormat`: Supply an input format to parse the key. Only the first value will be used.

4
website/.spelling
View File

@ -2262,4 +2262,6 @@ druid-momentsketch
druid-tdigestsketch
gce-extensions
prometheus-emitter
kubernetes-overlord-extensions
kubernetes-overlord-extensions
UCS
ISO646-US