Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/docs/content/development/extensions-core/kafka-eight-firehose.md

1.9 KiB
Raw Blame History

layout title
doc_page Apache Kafka Eight Firehose

Kafka Eight Firehose

To use this Apache Druid (incubating) extension, make sure to include druid-kafka-eight as an extension.

This firehose acts as a Kafka 0.8.x consumer and ingests data from Kafka.

Sample spec:

"firehose": {
  "type": "kafka-0.8",
  "consumerProps": {
    "zookeeper.connect": "localhost:2181",
    "zookeeper.connection.timeout.ms" : "15000",
    "zookeeper.session.timeout.ms" : "15000",
    "zookeeper.sync.time.ms" : "5000",
    "group.id": "druid-example",
    "fetch.message.max.bytes" : "1048586",
    "auto.offset.reset": "largest",
    "auto.commit.enable": "false"
  },
  "feed": "wikipedia"
}
property description required?
type This should be "kafka-0.8" yes
consumerProps The full list of consumer configs can be here. yes
feed Kafka maintains feeds of messages in categories called topics. This is the topic name. yes