This guide details the steps to integrate Synq with Databricks using the Unity Catalog feature. This integration enables efficient data observability and management across your Databricks environment.

Before proceeding, ensure you have:

  • Administrative access to your Databricks workspace
  • Permissions to manage Unity Catalog settings

⏱️ Estimated time to complete: 10 minutes.

​
Integration Overview

Integrating Synq with Databricks through Unity Catalog allows you to leverage Synq’s capabilities to monitor and manage data reliability and quality directly within your Databricks environment.

​
Prerequisites

  • Databricks Workspace URL: The URL of your Databricks workspace where Unity Catalog is configured.
  • Access Token: A Databricks access token with administrative privileges.
  • Warehouse ID: The identifier for the data warehouse within Databricks where Synq will monitor data.

​
Step-by-Step Guide

​
Step 1: Configuring Unity Catalog

  1. Log in to your Databricks workspace: Navigate to your Databricks workspace by entering the Workspace URL in your browser.

  2. Access the Unity Catalog: From the sidebar, select ‘Data’ and then ‘Unity Catalog’ to configure the data catalog settings.

  3. Create or Select a Catalog: Choose an existing catalog or create a new one where Synq will operate.

​
Step 2: Generate an Access Token

  1. Navigate to the User Settings: Click on your profile at the bottom left corner and select ‘User Settings’.

  2. Access Tokens: Go to the ‘Access Tokens’ tab and click on ‘Generate New Token’. Enter a description, set the expiration according to your policy, and note the generated token securely.

​
Step 3: Configure Synq Integration

  1. Login to Synq: Access your Synq dashboard.

  2. Add Databricks as an Integration: Navigate to ‘Data Sources’, select ‘Add integration’, and choose ‘Databricks’.

  3. Enter Integration Details: Provide the Workspace URL, Access Token, and Warehouse ID to establish the connection.

  4. Set Exclusion Rules: Define any exclusion rules for catalogs, schemas, and tables to tailor the monitoring to your needs.

​
Conclusion

Once configured, Synq will begin monitoring the specified data assets within Databricks, leveraging Unity Catalog for enhanced data management and observability.

For further assistance, contact our support team.

ClickHouseRedshift