Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
druid/website
History
Atul Mohan 03d6d395a0
Extension to read and ingest iceberg data files (#14329) 
This adds a new contrib extension: druid-iceberg-extensions which can be used to ingest data stored in Apache Iceberg format. It adds a new input source of type iceberg that connects to a catalog and retrieves the data files associated with an iceberg table and provides these data file paths to either an S3 or HDFS input source depending on the warehouse location.

Two important dependencies associated with Apache Iceberg tables are:

Catalog : This extension supports reading from either a Hive Metastore catalog or a Local file-based catalog. Support for AWS Glue is not available yet.
Warehouse : This extension supports reading data files from either HDFS or S3. Adapters for other cloud object locations should be easy to add by extending the AbstractInputSourceAdapter.
 		2023-07-18 08:59:57 +05:30
..
core fix: update footer copyright year (#12594) 2022-06-13 16:29:58 -07:00
pages/en Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
script Document how to report security issues (#13886) 2023-03-27 11:26:37 +05:30
scss Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00
static Tasks API documentation refactor (#14492) 2023-07-11 13:19:39 -07:00
.gitignore Docusaurus build framework + ingestion doc refresh. (#8311) 2019-08-20 21:48:59 -07:00
.spelling Extension to read and ingest iceberg data files (#14329) 2023-07-18 08:59:57 +05:30
README.md Updates to source and doc build pages (#11464) 2021-07-20 18:07:34 -07:00
package-lock.json remove website node-scss dep (#14275) 2023-05-17 04:10:46 -07:00
package.json remove website node-scss dep (#14275) 2023-05-17 04:10:46 -07:00
pom.xml fix website mvn build (#14458) 2023-06-22 12:14:23 -07:00
redirects.json Update Ingestion section (#14023) 2023-05-19 09:42:27 -07:00
sidebars.json API reference refactor (#14372) 2023-06-26 15:48:54 -07:00
siteConfig.js Docs: fix doc search (#13164) 2022-10-03 16:48:13 -07:00

README.md

Druid doc builder

This website was created with Docusaurus.

To view documentation run:

npm install

Then run:

npm start

The current version of the web site appears in your browser. Edit pages with your favorite editor. Refresh the web page after each edit to review your changes.

Dependencies

  • NodeJS. Use the version Docusaurus specifies, not a newer one. (For example, if 12.x is requested, don't install 16.x.) Docusaurus may require a version newer than that available in your Linux package repository, but older than the latest version. See this page to find the version required by Docusaurus.
  • The Yarn dependency from Docusaurus is optional. (This Yarn is not the Hadoop resource manager, it is a package manager for Node.js).
  • Docusaurus. Installed automatically as part of the the above npm commands.

Variables

Documentation pages can refer to a number of special variables using the {{var}} syntax:

  • DRUIDVERSION - the version of Druid in which the page appears. Allows creating links to files of the same version on GitHub.

The variables are not replaced when running the web site locally using the start command above.

Spellcheck

Please run a spellcheck before issuing a pull request to avoid a build failure due to spelling issues. Run:

npm run link-lint
npm run spellcheck

If you introduce new (correctly spelled) project names or technical terms, add them to the dictionary in the .spelling file in this directory. Also, terms enclosed in backticks are not spell checked. Example: `symbolName`