From bc66b42acc215bd0dc8d7fa23d653a7c87faf3be Mon Sep 17 00:00:00 2001 From: pdeva Date: Thu, 25 Sep 2014 01:48:45 -0700 Subject: [PATCH] FAQ on changing data granularity added cause this question is asked a ton of times on google group and i had a similar question and had to resort to the forums cause there was no doc. --- docs/content/Ingestion-FAQ.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/content/Ingestion-FAQ.md b/docs/content/Ingestion-FAQ.md index 59bb8fb7a93..fc83907c47a 100644 --- a/docs/content/Ingestion-FAQ.md +++ b/docs/content/Ingestion-FAQ.md @@ -42,6 +42,15 @@ You can check `:/druid/v2/datasources/?interva You can use IngestSegmentFirehose with index task to ingest existing druid segments using a new schema and change the name, dimensions, metrics, rollup, etc. of the segment. See [Firehose](Firehose.html) for more details on IngestSegmentFirehose. +## How can I change the granularity of existing data in Druid? + +In a lot of situations you may want to lower the granularity of older data. Example, any data older than 1 month has only hour level granularity but newer data has minute level granularity. + +To do this use the IngestSegmentFirehose and run an indexer task. The IngestSegment firehose will allow you to take in existing segments from Druid and aggregate them and feed them back into druid. It will also allow you to filter the data in those segments while feeding it back in. This means if there are rows you want to delete, you can just filter them away during re-ingestion. + +Typically the above will be run as a batch job to say everyday feed in a chunk of data and aggregate it. + + ## More information Getting data into Druid can definitely be difficult for first time users. Please don't hesitate to ask questions in our IRC channel or on our [google groups page](https://groups.google.com/forum/#!forum/druid-development).