mirror of
https://github.com/apache/druid.git
synced 2025-02-08 02:58:30 +00:00
a0234c4e13
There are two type of DeterminePartitionsJob: - When the input data is not assume grouped, there may be duplicate rows. In this case, two MR jobs are launched. The first one do group job to remove duplicate rows. And a second one to perform global sorting to find lower and upper bound for target segments. - When the input data is assume grouped, we only need to launch the global sorting MR job to find lower and upper bound for segments. Sampling strategy: - If the input data is assume grouped, sample by random at the mapper side of the global sort mr job. - If the input data is not assume grouped, sample at the mapper of the group job. Use hash on time and all dimensions and mod by sampling factor to sample, don't use random method because there may be duplicate rows.
Community Extensions
Please contribute all community extensions in this directory and include a doc of how your extension can be used under docs/development/extensions-contrib/.
Please note that community extensions are maintained by their original contributors and are not packaged with the core Druid distribution. If you'd like to take on maintenance for a community extension, please post on dev@druid.apache.org to let us know!