This guide explains how Coalesce Quality automatically links your dbt models with the Airflow tasks that execute them, providing end-to-end visibility from orchestration to data models.Prerequisites:
- Airflow integration configured
- dbt Core integration with
synq-dbtv2.0.0+ installed synq-dbtrunning within Airflow tasks- Valid integration token
How it works
When you run dbt withsynq-dbt inside an Airflow task, synq-dbt sends the Airflow context along with the artifacts. Coalesce Quality uses it to link each dbt run to the Airflow task run that started it.
You can then:
- See on a model page which Airflow tasks ran the model.
- See on an Airflow task run which dbt models it ran, with their status.
- See Airflow DAGs and tasks in the lineage view (Orchestration mode).
- See the DAG and task in issues for failed models.
Automatic context collection
Whensynq-dbt uploads your dbt artifacts, it also sends these Airflow environment variables if they are set:
Required for linking:
AIRFLOW_CTX_DAG_ID: DAG identifierAIRFLOW_CTX_TASK_ID: task identifierAIRFLOW_CTX_DAG_RUN_ID: DAG run identifier
AIRFLOW_CTX_TRY_NUMBER: try number of the task, so a retry links to the right try.
AIRFLOW_CTX_DAG_OWNER: DAG ownerAIRFLOW_CTX_EXECUTION_DATE: execution timestamp
Airflow sets these variables in the task process. Operators that run dbt in a container, such as
KubernetesPodOperator and DockerOperator, do not pass them on. Pass them with Jinja templates, as in the examples below.Setup verification
If you’ve already set up both integrations, the linking is already active. To verify:1. Check your Airflow setup
Ensure you’re usingsynq-dbt to execute dbt commands in your Airflow tasks:
Example with KubernetesPodOperator:
Regional Configuration: EU region customers (default) don’t need to set
SYNQ_API_ENDPOINT. US and AU region customers must add it to env_vars: "https://api.us.synq.io" or "https://api.au.synq.io".2. Run a dbt task in Airflow
Execute any DAG that runs dbt models usingsynq-dbt.
Artifacts and linking data typically appear within minutes of upload under normal conditions. Wait 5-10 minutes after your first execution for metadata to sync before checking.
3. View linked data
On model pages
Navigate to any dbt model page to see:- Airflow Execution History: Recent task executions that ran this model
- DAG Information: Which DAG and task executed the model
- Execution Timestamps: When the model was last executed via Airflow
On Airflow execution pages
Navigate to any Airflow task execution page to see:- Executed dbt Models: Complete list of all dbt models that ran in this task
- Model Run Status: Success/failure status for each model
- Execution Details: Runtime, rows affected, and other metrics per model
- Direct Links: Quick navigation to individual model pages
In lineage view
- Open Lineage
- Switch to Orchestration mode (toggle in the view settings)
- View your complete data pipeline including:
- Airflow DAGs and tasks
- dbt models executed by each task
- Connections between orchestration and data layers
Airflow context in issues
When a dbt model fails in an Airflow task, the issue shows the Airflow context of that run: the DAG ID, task ID, DAG run ID, execution date and DAG owner. Use it to find the task logs in Airflow and to route the issue to the DAG owner.Troubleshooting
Models not showing Airflow context
Check the following:- Using synq-dbt: Verify you’re running
synq-dbtinstead ofdbtin your Airflow tasks - synq-dbt version: Ensure you’re using v2.0.0 or later (run
synq-dbt --version) - Environment variables passed: Verify that required Airflow context variables (
AIRFLOW_CTX_DAG_ID,AIRFLOW_CTX_TASK_ID,AIRFLOW_CTX_DAG_RUN_ID) are being passed to your operator - Regional endpoint: US region customers must set
SYNQ_API_ENDPOINT=https://api.us.synq.io, AU customershttps://api.au.synq.io - Recent execution: Linking shows executions from the last 30 days
- Airflow version: Ensure your Airflow version sets the context variables (supported in Airflow 1.10+)
- Upload success: Check that
synq-dbtsuccessfully uploaded artifacts (check task logs for upload confirmation) - Network connectivity: Verify outbound HTTPS access to
developer.synq.io:443(EU),api.us.synq.io:443(US) orapi.au.synq.io:443(AU)
Lineage not showing orchestration
- Orchestration mode enabled: In Lineage view, ensure “Orchestration mode” is toggled on
- Airflow integration active: Verify your Airflow integration is properly configured
- Metadata sync: Wait 5-10 minutes after first execution for metadata to sync
Learn more
- Airflow integration guide: connect Airflow to Coalesce Quality
- dbt Core integration guide: install and configure synq-dbt
- synq-dbt GitHub repository: source code and advanced configuration