Skip to main content
This guide explains how Coalesce Quality automatically links your dbt models with the Airflow tasks that execute them, providing end-to-end visibility from orchestration to data models.Prerequisites:If you meet the prerequisites, linking works without extra setup.

How it works

When you run dbt with synq-dbt inside an Airflow task, synq-dbt sends the Airflow context along with the artifacts. Coalesce Quality uses it to link each dbt run to the Airflow task run that started it. You can then:
  • See on a model page which Airflow tasks ran the model.
  • See on an Airflow task run which dbt models it ran, with their status.
  • See Airflow DAGs and tasks in the lineage view (Orchestration mode).
  • See the DAG and task in issues for failed models.

Automatic context collection

When synq-dbt uploads your dbt artifacts, it also sends these Airflow environment variables if they are set: Required for linking:
  • AIRFLOW_CTX_DAG_ID: DAG identifier
  • AIRFLOW_CTX_TASK_ID: task identifier
  • AIRFLOW_CTX_DAG_RUN_ID: DAG run identifier
Recommended:
  • AIRFLOW_CTX_TRY_NUMBER: try number of the task, so a retry links to the right try.
Optional:
  • AIRFLOW_CTX_DAG_OWNER: DAG owner
  • AIRFLOW_CTX_EXECUTION_DATE: execution timestamp
Airflow sets these variables in the task process. Operators that run dbt in a container, such as KubernetesPodOperator and DockerOperator, do not pass them on. Pass them with Jinja templates, as in the examples below.
If Airflow starts every dbt run of a project, turn on Warn when a run arrives without its Airflow task in the advanced settings of the dbt Core integration. An upload without the three required variables then gets a warning, so a missing variable does not go unnoticed.

Setup verification

If you’ve already set up both integrations, the linking is already active. To verify:

1. Check your Airflow setup

Ensure you’re using synq-dbt to execute dbt commands in your Airflow tasks: Example with KubernetesPodOperator:
Regional Configuration: EU region customers (default) don’t need to set SYNQ_API_ENDPOINT. US and AU region customers must add it to env_vars: "https://api.us.synq.io" or "https://api.au.synq.io".
Example with DbtRunOperator:

2. Run a dbt task in Airflow

Execute any DAG that runs dbt models using synq-dbt.
Artifacts and linking data typically appear within minutes of upload under normal conditions. Wait 5-10 minutes after your first execution for metadata to sync before checking.

3. View linked data

On model pages

Navigate to any dbt model page to see:
  • Airflow Execution History: Recent task executions that ran this model
  • DAG Information: Which DAG and task executed the model
  • Execution Timestamps: When the model was last executed via Airflow

On Airflow execution pages

Navigate to any Airflow task execution page to see:
  • Executed dbt Models: Complete list of all dbt models that ran in this task
  • Model Run Status: Success/failure status for each model
  • Execution Details: Runtime, rows affected, and other metrics per model
  • Direct Links: Quick navigation to individual model pages

In lineage view

  1. Open Lineage
  2. Switch to Orchestration mode (toggle in the view settings)
  3. View your complete data pipeline including:
    • Airflow DAGs and tasks
    • dbt models executed by each task
    • Connections between orchestration and data layers

Airflow context in issues

When a dbt model fails in an Airflow task, the issue shows the Airflow context of that run: the DAG ID, task ID, DAG run ID, execution date and DAG owner. Use it to find the task logs in Airflow and to route the issue to the DAG owner.

Troubleshooting

Models not showing Airflow context

Check the following:
  1. Using synq-dbt: Verify you’re running synq-dbt instead of dbt in your Airflow tasks
  2. synq-dbt version: Ensure you’re using v2.0.0 or later (run synq-dbt --version)
  3. Environment variables passed: Verify that required Airflow context variables (AIRFLOW_CTX_DAG_ID, AIRFLOW_CTX_TASK_ID, AIRFLOW_CTX_DAG_RUN_ID) are being passed to your operator
  4. Regional endpoint: US region customers must set SYNQ_API_ENDPOINT=https://api.us.synq.io, AU customers https://api.au.synq.io
  5. Recent execution: Linking shows executions from the last 30 days
  6. Airflow version: Ensure your Airflow version sets the context variables (supported in Airflow 1.10+)
  7. Upload success: Check that synq-dbt successfully uploaded artifacts (check task logs for upload confirmation)
  8. Network connectivity: Verify outbound HTTPS access to developer.synq.io:443 (EU), api.us.synq.io:443 (US) or api.au.synq.io:443 (AU)

Lineage not showing orchestration

  1. Orchestration mode enabled: In Lineage view, ensure “Orchestration mode” is toggled on
  2. Airflow integration active: Verify your Airflow integration is properly configured
  3. Metadata sync: Wait 5-10 minutes after first execution for metadata to sync

Learn more