​

Data we collect

• Number of rows in every table in the monitored dataset(s)
• Timestamp of the last change of data in all tables in the monitored dataset(s)

​

Setup DWH Agent integration

1. In the app, navigate to Data Sources and click “Add integration”
2. Select “On-premise DWH Agent” from the list of available integrations
3. Enter a title for your integration (e.g., “Production DWH Agent”)
4. Click “Create”. You will receive:
   • client_id
   • client_secret
   Save these credentials - you’ll need them to configure the agent in the next section.

​

Agent Configuration

​

Example configuration:

# yaml-language-server: $schema=https://schemas.synq.io/synq-dwh/v1/config.schema.json
synq:
  client_id: "xxxx-xxxx-xxxx-xxx"
  client_secret: "${SYNQ_CLIENT_SECRET}"

connections:
  "pg-local":
    name: "PG Local"
    postgres:
      host: 1.2.3.4
      port: 54320
      username: postgres
      password: ${PG_PASSWORD}
      database: db
      allow_insecure: true

  "bq-nifty-motif-341212":
    bigquery:
      project_id: big-query-project-id
      region: europe-west2

  "ch-staging":
    name: Clickhouse
    clickhouse:
      host: host.clickhouse.cloud
      port: 9440
      instance_name: staging
      username: default
      password: ${STAGING_CLICKHOUSE_PASSWORD}

  "prod-snowflake":
    disabled: true
    snowflake:
      username: "SYNQ"
      password: "${SNOWFLAKE_DWH_PASSSWORD}"
      account: "snowflake-account"
      warehouse: "SYNQ_WH"
      databases: ["DB"]
      role: "SYNQ_ROLE"
      use_get_ddl: true

​

ClickHouse: naming the service and choosing a database

connections:
  "ch-prod":
    clickhouse:
      host: abc123xyz.europe-west4.gcp.clickhouse.cloud
      port: 9440
      instance_name: prod
      database: analytics
      username: quality
      password: ${CLICKHOUSE_PASSWORD}

• Give the same service the same name everywhere. It is the agent-side equivalent of Instance name on a Coalesce Quality-hosted ClickHouse integration, so a warehouse that moves between the two keeps its assets only if the name matches.
• Pick one and keep it. Changing it republishes every table under new paths, and the old ones stop being produced.

​

ClickHouse: cluster reads and server settings

connections:
  "ch-prod":
    clickhouse:
      host: clickhouse.internal
      port: 9440
      username: quality
      password: ${CLICKHOUSE_PASSWORD}
      cluster:
        mode: all_replicas        # or single_node
        name: analytics_cluster   # only applies to all_replicas
      settings:
        max_execution_time: "300"

• Omit cluster to read across a cluster named default. Every ClickHouse Cloud service provides one.
• mode: all_replicas with a name reads across the cluster you name. List what your server has with SELECT DISTINCT cluster FROM system.clusters. This read needs GRANT REMOTE ON *.*.
• mode: single_node reads system tables directly on the connected node. This is the only setting that works on an install whose remote_servers defines no cluster, and it needs no REMOTE grant. Choose it only when the server really is one node: on a service with replicas it reports the metadata of whichever node answered, which looks like a successful but incomplete scrape. name does not apply.