Skip to main content
Every number in Analytics and in a data product’s quality score is built from the same two counts:
  • Tests: the distinct checks that ran at least once in a period.
  • Tests with issues: the distinct checks that had an open issue at any point in that period.
This page explains what goes into those two counts, how checks are grouped into data quality dimensions, and how each metric is derived from them.

​
What counts as a test

A test is any check that produces runs in Coalesce Quality:
  • dbt tests, and dbt model, source, snapshot, seed and metric runs
  • SQLMesh audits and model runs
  • Coalesce Transform node runs and node tests
  • Coalesce Quality monitors, including segmented monitors
  • SQL tests and reconciliations
A test is counted in a period if it ran at least once in that period, whatever the result. A test that ran 50 times in a week counts as one test for that week.

​
When a test has an issue

A test has an issue in a period when one of its issues was open at any point during that period. An issue is open from the run that raised it until the run that resolved it, so it is counted in every period it spans:
  • an issue that started in the period
  • an issue that was resolved in the period
  • an issue that started earlier and is still open
Runs with a warning, error or critical status raise issues. A warning counts as an issue, so a test whose severity is set to warn still lowers your scores while it is failing. Issues whose status is set to Expected or No action needed are not counted. A test is counted once per period no matter how many issues it had.

​
Data quality dimensions

Each test is assigned to one dimension based on its type. Tests whose type is not listed fall into Uncategorized.
Model, source and node runs count under Timeliness. A dbt model that fails to build counts as a Timeliness test with an issue.

​
Analytics metrics

Analytics splits the selected date range into weekly or monthly periods. Each column in a table is one period, and each row is one dimension. Every metric is calculated per period and per dimension. Values from different periods are never added together. A period in which no test in a dimension ran shows — for that dimension. The Total row adds up the counts of every dimension first, then calculates the percentage from those sums. It is not an average of the dimension percentages, so a dimension with many tests moves the total more than a dimension with few.

​
Choosing what is counted

By default Analytics counts every test in the workspace. Filters narrow it to tests on matching assets. A test is included if any asset it checks matches the filter, so a relationships test between two tables counts when either table matches. The Product filter includes the data product’s own assets. Turn on Include upstream assets to also count every test on assets upstream of the product.

​
Data product quality score

The data product Overview shows a quality score and test coverage, calculated from the same counts as Analytics over the last 4 weeks, including the current week. The score covers tests on the data product’s assets and on every asset upstream of them, so a failing test on a source table lowers the score of every product built on it. Quality score
The averages are taken over the 4 weeks before dividing, and the result is rounded to a whole percent. The score is — when no test ran in those 4 weeks. The score is shown in:
  • Green at 100%
  • Amber from 90% to 99%
  • Red below 90%
Test coverage is the average number of tests that ran per week over the 4 weeks, rounded to a whole number. It is a count of tests, not a percentage of assets. Change indicators compare the most recent week with the week before it. The score change uses each week’s own percentage. The coverage change is the difference in the number of tests that ran. Breakdown by dimension. When tests from more than one dimension ran, click the score to see the same calculation for each dimension that had tests in the last 4 weeks.