Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #654 from metamx/ingest-firehose-doc 

doc for IngestSegmentFirehose
This commit is contained in:
fjy 2014-08-01 10:31:12 -06:00
parent d202eae3b5 80a4f7c5b5
commit fd0c738221
2 changed files with 36 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
docs/content/Firehose.md
View File

@ -36,10 +36,40 @@ See [Examples](Examples.html). This firehose connects directly to the twitter sp
See [Examples](Examples.html). This firehose creates a stream of random numbers. See [Examples](Examples.html). This firehose creates a stream of random numbers.
#### RabbitMqFirehouse #### RabbitMqFirehose
This firehose ingests events from a define rabbit-mq queue. This firehose ingests events from a define rabbit-mq queue.
#### IngestSegmentFirehose
This Firehose can be used to read the data from existing druid segments.
It can be used ingest existing druid segments using a new schema and change the name, dimensions, metrics, rollup, etc. of the segment.
A sample ingest firehose spec is shown below -
```json
{
"type" : "ingestSegment",
"dataSource" : "wikipedia",
"interval" : "2013-01-01/2013-01-02",
"dimensions":[],
"metrics":[]
}
```
|property|description|required?|
|--------|-----------|---------|
|type|ingestSegment. Type of firehose|yes|
|dataSource|A String defining the data source to fetch rows from, very similar to a table in a relational database|yes|
|interval|A String representing ISO-8601 Interval. This defines the time range to fetch the data over.|yes|
|dimensions|The list of dimensions to select. If left empty, all dimensions are selected.|no|
|metrics|The list of metrics to select. If left empty, all metrics are returned.|no|
|filter| See [Filters](Filters.html)|yes|
Parsing Data Parsing Data
------------ ------------

5
docs/content/Ingestion-FAQ.md
View File

@ -37,6 +37,11 @@ You can check the coordinator console located at `<COORDINATOR_IP>:<PORT>/cluste
You can check `<BROKER_IP>:<PORT>/druid/v2/datasources/<YOUR_DATASOURCE>?interval=0/3000` for the dimensions and metrics that have been created for your datasource. Make sure that the name of the aggregators you use in your query match one of these metrics. Also make sure that the query interval you specify match a valid time range where data exists. Note: the broker endpoint will only return valid results on historical segments. You can check `<BROKER_IP>:<PORT>/druid/v2/datasources/<YOUR_DATASOURCE>?interval=0/3000` for the dimensions and metrics that have been created for your datasource. Make sure that the name of the aggregators you use in your query match one of these metrics. Also make sure that the query interval you specify match a valid time range where data exists. Note: the broker endpoint will only return valid results on historical segments.
## How can I Reindex existing data in Druid with schema changes?
You can use IngestSegmentFirehose with index task to ingest existing druid segments using a new schema and change the name, dimensions, metrics, rollup, etc. of the segment.
See [Firehose](Firehose.html) for more details on IngestSegmentFirehose.
## More information ## More information
Getting data into Druid can definitely be difficult for first time users. Please don't hesitate to ask questions in our IRC channel or on our [google groups page](https://groups.google.com/forum/#!forum/druid-development). Getting data into Druid can definitely be difficult for first time users. Please don't hesitate to ask questions in our IRC channel or on our [google groups page](https://groups.google.com/forum/#!forum/druid-development).