d007477742
* Docusaurus build framework + ingestion doc refresh. * stick to npm instead of yarn * fix typos * restore some _bin * Adjustments. * detect and fix redirect anchors * update anchor lint * Web-console: remove specific column filters (#8343) * add clear filter * update tool kit * remove usless check * auto run * add % * Fix resource leak (#8337) * Fix resource leak * Patch comments * Enable Spotbugs NP_NONNULL_RETURN_VIOLATION (#8234) * Fixes from PR review. * Fix more anchors. * Preamble nix. * Fix more anchors, headers * clean up placeholder page * add to website lint to travis config * better broken link checking * travis fix * Fixed more broken links * better redirects * unfancy catch * fix LGTM error * link fixes * fix md issues * Addl fixes
4.1 KiB
| id | title |
|---|---|
| metadata-migration | Metadata Migration |
If you have been running an evaluation Druid cluster using the built-in Derby metadata storage and wish to migrate to a more production-capable metadata store such as MySQL or PostgreSQL, this document describes the necessary steps.
Shut down cluster services
To ensure a clean migration, shut down the non-coordinator services to ensure that metadata state will not change as you do the migration.
When migrating from Derby, the coordinator processes will still need to be up initially, as they host the Derby database.
Exporting metadata
Druid provides an Export Metadata Tool for exporting metadata from Derby into CSV files which can then be imported into your new metadata store.
The tool also provides options for rewriting the deep storage locations of segments; this is useful for deep storage migration.
Run the export-metadata tool on your existing cluster, and save the CSV files it generates. After a successful export, you can shut down the coordinator.
Initializing the new metadata store
Create database
Before importing the existing cluster metadata, you will need to set up the new metadata store.
The MySQL extension and PostgreSQL extension docs have instructions for initial database setup.
Update configuration
Update your Druid runtime properties with the new metadata configuration.
Create Druid tables
Druid provides a metadata-init tool for creating Druid's metadata tables. After initializing the Druid database, you can run the commands shown below from the root of the Druid package to initialize the tables.
In the example commands below:
libis the the Druid lib directoryextensionsis the Druid extensions directorybasecorresponds to the value ofdruid.metadata.storage.tables.basein the configuration,druidby default.- The
--connectURIparameter corresponds to the value ofdruid.metadata.storage.connector.connectURI. - The
--userparameter corresponds to the value ofdruid.metadata.storage.connector.user. - The
--passwordparameter corresponds to the value ofdruid.metadata.storage.connector.password.
MySQL
cd ${DRUID_ROOT}
java -classpath "lib/*" -Dlog4j.configurationFile=conf/druid/cluster/_common/log4j2.xml -Ddruid.extensions.directory="extensions" -Ddruid.extensions.loadList=[\"mysql-metadata-storage\"] -Ddruid.metadata.storage.type=mysql org.apache.druid.cli.Main tools metadata-init --connectURI="<mysql-uri>" --user <user> --password <pass> --base druid
PostgreSQL
cd ${DRUID_ROOT}
java -classpath "lib/*" -Dlog4j.configurationFile=conf/druid/cluster/_common/log4j2.xml -Ddruid.extensions.directory="extensions" -Ddruid.extensions.loadList=[\"postgresql-metadata-storage\"] -Ddruid.metadata.storage.type=postgresql org.apache.druid.cli.Main tools metadata-init --connectURI="<postgresql-uri>" --user <user> --password <pass> --base druid
Import metadata
After initializing the tables, please refer to the import commands for your target database.
Restart cluster
After importing the metadata successfully, you can now restart your cluster.