Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/website
History
Gian Merlino 227b57dd8e
Compaction: Fetch segments one at a time on main task; skip when possible. (#13280) 
* Compaction: Fetch segments one at a time on main task; skip when possible.

Compact tasks include the ability to fetch existing segments and determine
reasonable defaults for granularitySpec, dimensionsSpec, and metricsSpec.
This is a useful feature that makes compact tasks work well even when the
user running the compaction does not have a clear idea of what they want
the compacted segments to be like.

However, this comes at a cost: it takes time, and disk space, to do all
of these fetches. This patch improves the situation in two ways:

1) When segments do need to be fetched, download them one at a time and
   delete them when we're done. This still takes time, but minimizes the
   required disk space.

2) Don't fetch segments on the main compact task when they aren't needed.
   If the user provides a full granularitySpec, dimensionsSpec, and
   metricsSpec, we can skip it.

* Adjustments.

* Changes from code review.

* Fix logic for determining rollup.
 		2022-11-07 14:50:14 +05:30
..
core fix: update footer copyright year (#12594) 2022-06-13 16:29:58 -07:00
i18n Various documentation updates. (#13107) 2022-09-16 21:58:11 -07:00
pages/en Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
script Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00
scss Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00
static Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00
.gitignore Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
.spelling Compaction: Fetch segments one at a time on main task; skip when possible. (#13280) 2022-11-07 14:50:14 +05:30
README.md Updates to source and doc build pages (#11464) 2021-07-20 18:07:34 -07:00
package-lock.json Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00
package.json Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00
pom.xml Bump up the version to 25.0.0 (#12975) 2022-08-29 11:27:38 +05:30
redirects.json Various documentation updates. (#13107) 2022-09-16 21:58:11 -07:00
sidebars.json Various documentation updates. (#13107) 2022-09-16 21:58:11 -07:00
siteConfig.js Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00

README.md

Druid doc builder

This website was created with Docusaurus.

To view documentation run:

npm install

Then run:

npm start

The current version of the web site appears in your browser. Edit pages with your favorite editor. Refresh the web page after each edit to review your changes.

Dependencies

  • NodeJS. Use the version Docusaurus specifies, not a newer one. (For example, if 12.x is requested, don't install 16.x.) Docusaurus may require a version newer than that available in your Linux package repository, but older than the latest version. See this page to find the version required by Docusaurus.
  • The Yarn dependency from Docusaurus is optional. (This Yarn is not the Hadoop resource manager, it is a package manager for Node.js).
  • Docusaurus. Installed automatically as part of the the above npm commands.

Variables

Documentation pages can refer to a number of special variables using the {{var}} syntax:

  • DRUIDVERSION - the version of Druid in which the page appears. Allows creating links to files of the same version on GitHub.

The variables are not replaced when running the web site locally using the start command above.

Spellcheck

Please run a spellcheck before issuing a pull request to avoid a build failure due to spelling issues. Run:

npm run link-lint
npm run spellcheck

If you introduce new (correctly spelled) project names or technical terms, add them to the dictionary in the .spelling file in this directory. Also, terms enclosed in backticks are not spell checked. Example: `symbolName`