druid-docs-cn/development/extensions-core/postgresql.md

5.2 KiB

id title
postgresql PostgreSQL Metadata Store

To use this Apache Druid extension, make sure to include postgresql-metadata-storage as an extension.

Setting up PostgreSQL

  1. Install PostgreSQL

Use your favorite package manager to install PostgreSQL, e.g.:

  • on Ubuntu/Debian using apt apt-get install postgresql
  • on OS X, using Homebrew brew install postgresql
  1. Create a druid database and user

On the machine where PostgreSQL is installed, using an account with proper postgresql permissions:

Create a druid user, enter diurd when prompted for the password.

createuser druid -P

Create a druid database owned by the user we just created

createdb druid -O druid

Note: On Ubuntu / Debian you may have to prefix the createuser and createdb commands with sudo -u postgres in order to gain proper permissions.

  1. Configure your Druid metadata storage extension:

Add the following parameters to your Druid configuration, replacing <host> with the location (host name and port) of the database.

druid.extensions.loadList=["postgresql-metadata-storage"]
druid.metadata.storage.type=postgresql
druid.metadata.storage.connector.connectURI=jdbc:postgresql://<host>/druid
druid.metadata.storage.connector.user=druid
druid.metadata.storage.connector.password=diurd

Configuration

In most cases, the configuration options map directly to the postgres JDBC connection options.

Property Description Default Required
druid.metadata.postgres.ssl.useSSL Enables SSL false no
druid.metadata.postgres.ssl.sslPassword The Password Provider or String password for the client's key. none no
druid.metadata.postgres.ssl.sslFactory The class name to use as the SSLSocketFactory none no
druid.metadata.postgres.ssl.sslFactoryArg An optional argument passed to the sslFactory's constructor none no
druid.metadata.postgres.ssl.sslMode The sslMode. Possible values are "disable", "require", "verify-ca", "verify-full", "allow" and "prefer" none no
druid.metadata.postgres.ssl.sslCert The full path to the certificate file. none no
druid.metadata.postgres.ssl.sslKey The full path to the key file. none no
druid.metadata.postgres.ssl.sslRootCert The full path to the root certificate. none no
druid.metadata.postgres.ssl.sslHostNameVerifier The classname of the hostname verifier. none no
druid.metadata.postgres.ssl.sslPasswordCallback The classname of the SSL password provider. none no
druid.metadata.postgres.dbTableSchema druid meta table schema public no

PostgreSQL Firehose

The PostgreSQL extension provides an implementation of an SqlFirehose which can be used to ingest data into Druid from a PostgreSQL database.

{
  "type": "index_parallel",
  "spec": {
    "dataSchema": {
      "dataSource": "some_datasource",
      "parser": {
        "parseSpec": {
          "format": "timeAndDims",
          "dimensionsSpec": {
            "dimensionExclusions": [],
            "dimensions": [
              "dim1",
              "dim2",
              "dim3"
            ]
          },
          "timestampSpec": {
            "format": "auto",
            "column": "ts"
          }
        }
      },
      "metricsSpec": [],
      "granularitySpec": {
        "type": "uniform",
        "segmentGranularity": "DAY",
        "queryGranularity": {
          "type": "none"
        },
        "rollup": false,
        "intervals": null
      },
      "transformSpec": {
        "filter": null,
        "transforms": []
      }
    },
    "ioConfig": {
      "type": "index_parallel",
      "firehose": {
        "type": "sql",
        "database": {
          "type": "postgresql",
          "connectorConfig": {
            "connectURI": "jdbc:postgresql://some-rds-host.us-west-1.rds.amazonaws.com:5432/druid",
            "user": "admin",
            "password": "secret"
          }
        },
        "sqls": [
          "SELECT * FROM some_table"
        ]
      }
    },
    "tuningconfig": {
      "type": "index_parallel"
    }
  }
}