Skip to main content
This guide shows you how to connect your dbt Core project to Coalesce Quality to track model runs, test results, and metadata changes.Prerequisites:
  • Admin access to your Coalesce Quality workspace
  • Ability to modify your dbt orchestration tool (Airflow, GitHub Actions, etc.)
⏱️ Estimated time: 15 minutes
Using dbt Cloud? You can integrate directly through Settings → Integrations → Add Integration → dbt Cloud instead of following this guide.

Set up dbt Core integration

Create integration in Coalesce Quality

  1. Navigate to Settings → Integrations → Add Integration
  2. Select dbt Core from the integration options
The setup has three steps. Settings: Enter a descriptive integration name (e.g., Production dbt Core). If you use Snowflake, Databricks or Trino, you can also enter the Snowflake account, Databricks workspace URL or Trino instance. Use the same value as in your warehouse integration, so that models link to their tables. For ClickHouse, enter a default database name. Git integration: Select the Git integration for the repository with your dbt project, so that model changes link to commits. Enter the branch name, and the path to the dbt project if it is not in the repository root (e.g., analytics/dbt/). Advanced: Leave the defaults unless you need one of the options described in Advanced settings. After you click Create, the integration page opens.

Manage integration tokens

synq-dbt authenticates with an upload token. Create one in the Upload tokens section of the integration page. You can also reach it from Settings → Integrations: open the menu of your dbt Core integration and click Manage tokens.
dbt Core token management interface
From the token management screen, you can:
  • Create new tokens, for example one per environment
  • Invalidate compromised tokens
  • Copy token snippets for easy integration
List of dbt Core integration tokens
Use one dbt Core integration per dbt project. A new integration locks itself to the dbt project of its first upload, and rejects uploads from any other project. See Advanced settings.

Install synq-dbt

About synq-dbt

synq-dbt is a command-line wrapper around dbt Core. It runs your dbt command with the same arguments, then uploads the artifacts to Coalesce Quality. It works with any dbt Core version, and you can use it anywhere you run dbt, for example in Airflow, GitHub Actions or Dagster. Collected artifacts:
  • manifest.json: Project structure and dependencies
  • run_results.json: Execution status and performance metrics
  • catalog.json: Complete data warehouse schema information
  • sources.json: Source freshness test results
catalog.json is only written by dbt docs generate. dbt run, dbt test and dbt build don’t produce it. See Send catalog.json below. How it works:
  1. Runs your installed dbt with all the arguments you gave it
  2. Reads the artifacts from the dbt target directory
  3. Uploads them to Coalesce Quality with the token from SYNQ_TOKEN, retrying up to three times if the upload fails
  4. Exits with the dbt exit code, also when the upload fails, so your pipeline behaves the same as with plain dbt

Installation methods

Choose the installation method that matches your dbt orchestration setup:

Airflow with a container operator

  1. Set the token: In the Airflow UI, create a variable SYNQ_TOKEN with your integration token.
  2. Update Dockerfile:
latest/download always resolves to the newest release. To pin an image to a known version instead, replace it with download/v2.1.0. The tags are on the releases page.
  1. Update your operator (the example uses KubernetesPodOperator, DockerOperator works the same way):
Linking dbt models to Airflow tasks: To automatically link your dbt models with the Airflow tasks that execute them, see the Airflow + dbt Core Linking guide. This enables bidirectional visibility between your orchestration and data layers.

Airflow with dbt Plugin

  1. Set the token: Create a variable SYNQ_TOKEN in the Airflow UI.
  2. Install synq-dbt:
  1. Update DbtOperator:
For linking dbt models to Airflow tasks, see the Airflow + dbt Core Linking guide.

Dagster

  1. Configure environment: Add SYNQ_TOKEN=<your-token> to your .env file. For US region workspaces also add SYNQ_API_ENDPOINT=https://api.us.synq.io, and for AU https://api.au.synq.io.
  2. Update resources in definitions.py:
  1. Update assets in assets.py:

Docker

Add to your Dockerfile:
Swap latest/download for download/v2.1.0 if you would rather the image pin a version.

Linux/macOS

Download and install:
Binaries are also published for Linux on ARM (synq-dbt-arm64-linux) and macOS on Intel (synq-dbt-amd64-darwin).

Use synq-dbt

Basic usage

Replace your existing dbt Core commands with synq-dbt:
All dbt Core arguments and options work exactly the same way.

Send catalog.json

None of the commands above write catalog.json. Add a separate docs generate step to the same job or DAG, after run or build:
synq-dbt docs generate runs dbt docs generate, which queries your warehouse and writes catalog.json to the target directory. synq-dbt then uploads it along with the other artifacts. With Airflow or Dagster, add docs generate as another synq-dbt command, the same way you run build. If you run plain dbt docs generate instead, upload the catalog afterwards from the same working directory with synq-dbt synq_upload_artifacts.
If an upload doesn’t include catalog.json, Coalesce Quality reuses the last catalog it received, but only if it is at most 7 days older than the upload’s manifest.json. If there is no such catalog, the models are published without column types and descriptions, and the upload gets a recommendation to run dbt docs generate. After more than 30 days without a new catalog, the column information from the last one is no longer treated as current. It is replaced wherever the warehouse describes the same table.
If you deliberately don’t send catalog.json, turn on Skip catalog.json warnings in the integration’s advanced settings. This only hides the recommendation. The 7-day and 30-day rules still apply.

Upload existing artifacts

If you have already generated dbt artifacts and want to upload them to Coalesce Quality:
Include dbt logs:

Configuration options

Environment variables:
  • SYNQ_TOKEN: your integration token (required)
  • SYNQ_API_ENDPOINT: API endpoint for your region (required outside EU)
  • SYNQ_TARGET_DIR: directory to read artifacts from. If you don’t set it, synq-dbt uses the --target-path flag of the dbt command, then DBT_TARGET_PATH, then target-path in dbt_project.yml, then target.
  • SYNQ_DBT_BIN: name or path of the dbt executable to run (default: dbt)
Regional configuration:
EU region customers (default) don’t need to set SYNQ_API_ENDPOINT. US and AU region customers must configure it.
For US region workspaces, set the API endpoint:
For AU region workspaces:
For EU region workspaces, synq-dbt uses https://developer.synq.io by default. Network requirements:
  • Allow outbound HTTPS traffic to developer.synq.io:443 (EU), api.us.synq.io:443 (US) or api.au.synq.io:443 (AU)
Artifacts usually appear in Coalesce Quality a few minutes after the upload. If all upload attempts fail, synq-dbt logs the error. To send the artifacts again, run synq-dbt synq_upload_artifacts from the same directory.

Advanced settings

The Advanced section of the integration page has these options:
  • Skip catalog.json warnings: hides the recommendation to run dbt docs generate. See Send catalog.json above.
  • Skip Git connection warnings: hides the warning shown when no Git integration is selected.
  • Ignore missing manifest.json warnings: hides the warning for an upload without manifest.json. Coalesce Quality uses the last uploaded manifest.json in that case.
  • Warn when a run arrives without its Airflow task: turn this on if Airflow starts every dbt run of this project. An upload without the Airflow context variables then gets a warning. See Airflow + dbt Core Linking.
  • Include integration name in asset folders: adds the integration name to asset folders. Use it when the same model is imported into more than one dbt Core project.

dbt project handling

A dbt Core integration serves one dbt project. If several projects upload to the same integration, for example because they share a token, each upload replaces the assets of the previous project. Create a separate integration for each project. These options control how the integration handles project names:
  • Namespace assets by dbt project name: the asset paths use the dbt project name instead of the integration ID. On by default for new integrations. If you turn it on for an existing integration, its dbt asset paths change and open issues on dbt models start again.
  • Lock the integration to a single dbt project: the integration saves the project name of its first upload as the required project name. It then rejects uploads from any other project. On by default for new integrations.
  • Required dbt project name: the only dbt project the integration accepts. Leave it empty to let the lock fill it in from the first upload.
  • Asset identifier override: an explicit namespace for the assets of this integration. It replaces both the project name and the integration ID. Use it to keep the same dbt project apart when it uploads to two integrations, for example prod and qa.
For advanced configuration options and troubleshooting, see the synq-dbt GitHub repository.