This guide shows you how to connect your Coalesce Transform workspace to SYNQ to track pipeline runs, job statuses, and metadata changes.Prerequisites:
- Admin access to your SYNQ workspace
- Admin access to your Coalesce Transform workspace
- Ability to generate API tokens in Coalesce
About Coalesce Transform
Coalesce is a metadata-driven data transformation platform that helps data teams build and operate governed data pipelines on cloud data platforms. It combines visual DAG-based design with code editing capabilities, built-in governance, and column-level lineage tracking. Integrating Coalesce Transform with SYNQ enables you to:- Monitor data pipeline health and execution status
- Track metadata changes across your transformation workflows
- Visualize data lineage from Coalesce pipelines to downstream assets
- Set up alerts on pipeline failures or data quality issues
- Correlate transformation runs with data warehouse changes
SYNQ uses read-only APIs to fetch metadata from Coalesce. The integration cannot modify your Coalesce projects, configurations, or pipelines.
Set up Coalesce Transform integration
Step 1: Create integration in SYNQ
- Navigate to Settings → Integrations → Add Integration
- Select Coalesce Transform from the integration options
Step 2: Configure integration settings
Integration name
Enter a descriptive name for your integration (e.g.,Production Coalesce or Coalesce Transform)
API endpoint
Select your Coalesce server region from the dropdown. Coalesce operates across multiple regions:- US primary -
https://app.us1.coalesce.io - US secondary -
https://app.us2.coalesce.io - EU primary -
https://app.eu1.coalesce.io - EU secondary -
https://app.eu2.coalesce.io - Canada -
https://app.ca1.coalesce.io - Australia -
https://app.au1.coalesce.io - India -
https://app.in1.coalesce.io - UAE -
https://app.ae1.coalesce.io - UK -
https://app.uk1.coalesce.io - Singapore -
https://app.sg1.coalesce.io
API token
Generate an access token in your Coalesce Transform workspace:- Log into your Coalesce workspace
- Navigate to the Deploy section
- Click Generate Access Token
- Copy the generated token immediately (you won’t be able to view it again)
- Paste it into the API Token field in SYNQ
Your Coalesce account must have API access enabled. The generated token will have the same access rights as the user who created it, so ensure you have access to the projects you want to sync with SYNQ.
Project selection
After entering valid credentials, SYNQ will automatically load your available Coalesce projects. You can:- Select specific projects by checking the boxes next to project names. Each project card displays:
- Project name and description
- Associated environment tags (Production, Staging, Development, etc.)
- Sync all projects by leaving all checkboxes unchecked. This is useful when you want SYNQ to automatically include new projects as they’re created in Coalesce.
Recommendation: If you have many projects, start by selecting only production projects. You can always edit the integration later to include additional projects.
Fetch interval
Configure how often SYNQ should poll Coalesce for updates. The default is 5 minutes. You can adjust this based on:- Higher frequency (1-5 minutes): For production environments where you need near real-time monitoring
- Lower frequency (15-30 minutes): For development environments or when API rate limits are a concern
Step 3: Complete integration setup
- Review your configuration settings
- Click Create Integration
- SYNQ will immediately begin syncing metadata from your Coalesce workspace
Verify integration
After creating the integration, you can verify it’s working correctly:- Navigate to Settings → Integrations
- Find your Coalesce Transform integration in the list
- Check the Last Sync timestamp to confirm data is being fetched
- Navigate to Lineage to see Coalesce nodes and their relationships
- Check that your Coalesce projects appear in the asset catalog
Initial synchronization may take several minutes depending on the size of your Coalesce workspace. Subsequent syncs will be incremental and faster.
What data is synced?
When you connect Coalesce Transform to SYNQ, the following metadata is synchronized:- Projects: All selected Coalesce projects and their configurations
- Environments: Development, staging, production, and custom environments
- Nodes: Sources, stages, dimensions, facts, views, and custom node types
- Job runs: Execution history, status, duration, and error messages
- Data lineage: Column-level lineage between nodes and downstream assets
- Storage mappings: Environment-specific database and schema configurations
- Metadata changes: Schema changes, node modifications, and deployment history
- Provide comprehensive lineage visualization
- Monitor pipeline execution health
- Alert on failures or anomalies
- Track data quality metrics across transformation workflows
Managing your integration
Edit integration settings
To modify your Coalesce Transform integration:- Navigate to Settings → Integrations
- Find your Coalesce Transform integration
- Click Edit
- Update settings:
- Change selected projects
- Adjust fetch interval
- Update API token (if rotated)
- Switch regions (requires new API token)
Troubleshooting
Projects not loading
If projects don’t appear after entering your credentials:- Verify your API token is correct
- Ensure the API endpoint matches your Coalesce region
- Check that your token has the required read permissions
- Try regenerating the token in Coalesce with appropriate permissions
Integration shows “Failed” status
Common causes and solutions:- Invalid credentials: Regenerate your API token in Coalesce and update the integration
- Token expired: If your organization enforces token expiration, generate a new token
- Network issues: Ensure SYNQ can reach the Coalesce API endpoint (check firewalls if applicable)
- Permission changes: Verify the token still has read access to projects and environments
Missing metadata or lineage
If some Coalesce data isn’t appearing in SYNQ:- Check that the affected projects are selected in the integration configuration
- Verify the projects have been deployed and have run history in Coalesce
- Wait for the next sync cycle (based on your fetch interval)
- Check the integration logs in Settings → Integrations → [Your Integration] → View Logs
Support
If you encounter issues or need assistance with your Coalesce Transform integration:- Check the SYNQ Support documentation
- Contact SYNQ support through your shared Slack channel
- Reach out to the SYNQ team at [email protected]