The freshness monitor looks at a table’s timestamp and alerts you if it’s been too long since the last update.
Freshness monitors rely on your data warehouse metadata tables for timestamp information (e.g., Snowflake table information schema). As the monitor queries the information schema, you only incur minimal additional costs even when they are run frequently. The monitor works out of the box, which makes it easy to deploy at scale.
Default freshness monitors are run every 60 minutes.
Setting up a freshness monitor
- Navigate to
Create monitor groupto define the tables you want to monitor
- Use the Synq browser to narrow down the tables you want to monitor
- Browser—select specific schemas or search for tables to monitor (Synq automatically maps your data warehouse tables and dbt models)
- Annotation—select assets with metadata definitions such as tag defined in a yml file from dbt
- Important—select assets that you’ve marked as important
- Query—advanced selection. E.g., search for specific keyword matches
To deploy freshness monitors on all dbt sources automatically, filter by
Freshnessto set up a freshness monitor group
- Name the monitor (e.g., freshness on all sources).
continueto set up the monitor
You’ll be able to see the monitors on the Health overview page and adjust the monitor sensitivity.
As soon as you’ve set the monitor up, it will start learning the behaviors of your data every hour. You should expect ~10 days before the monitor has built an understanding of the seasonality and patterns in your data and is confidently able to predict anomalies