Running locally - Coalesce Quality Documentation

The local loop is where a suite gets written. It runs against credentials on your machine, against databases your machine can reach, and it is the fastest way to iterate.

The three commands

run-check

Stage one only: counts and checksums. One query per side.

run-drill

Stage two only: locate the differences. Assumes you already know there are some.

run --auto-drill

Stage one, then stage two on whatever mismatched. The everyday command.

synq-recon run-check suite.yaml                        # is anything different?
synq-recon run-drill  suite.yaml --include orders      # where is it?
synq-recon run        suite.yaml --auto-drill          # both, in one pass

Reconciliation names are selected with --include and --exclude, both repeatable — not positionally. run-drill suite.yaml orders is rejected with accepts 1 arg(s), received 2.

Selecting and parameterising a run

Flag	Effect
`--include <name>`	Run only these reconciliations. Repeatable.
`--exclude <name>`	Run everything except these. Repeatable.
`--var key=value`	Override a template variable. Repeatable.
`--concurrency <n>`	Reconciliations to run in parallel. Default `1`.
`--timeout <duration>`	Per-query timeout. Default `5m`.
`--connections <path>`	Connections file. `.connections.yaml` in the working directory is picked up automatically.
`--dbt-profiles <path>`	Resolve connections from a dbt `profiles.yml` as a fallback.

--concurrency fans reconciliations onto their connections; each connection additionally caps its own parallel queries with parallelism: (default 8), so raising concurrency does not stampede a single warehouse. Every flag on every command is in the CLI reference, which is generated from the binary — this page covers the ones with a decision attached.

Reading the output

synq-recon run-check suite.yaml -o json

Set -o explicitly whenever something is going to parse the output. The default is table for a human terminal, but it switches to toon when the CLI detects an AI-agent environment. An agent that assumed table — or json — gets neither.

Three more things to know before you pipe anything:

Logs go to stderr, results to stdout

Redirect with 2>/dev/null. Never 2>&1 — that merges log lines into the document you are trying to parse.

synq-recon run-check suite.yaml -o json 2>/dev/null | jq '.summary'

run / run-check / run-drill render only two shapes

-o json, and a text report for every other value. The output-shaping flags — --jq, --columns, --no-headers, --wide — do not apply to them; pipe their -o json through jq yourself.The workspace commands (suite, deployment, runs, audit-logs, connections, auth) do honour all six formats and all four flags.

run --auto-drill is not a single JSON document

It emits the quick-check document, a plain-text banner, then one drill document per drilled reconciliation. To parse results, run the two stages separately — each is one valid document:

synq-recon run-check suite.yaml -o json 2>/dev/null > check.json
synq-recon run-drill suite.yaml -o json 2>/dev/null > drill.json

Useful one-liners:

# names of the reconciliations that differ
synq-recon run-check suite.yaml -o json 2>/dev/null \
  | jq -r '.results[] | select(.match == false) | .name'

# key ranges where a difference lives
synq-recon run-drill suite.yaml --include orders -o json 2>/dev/null \
  | jq -r '.mismatch_leaves[] | "\(.segment.min_key) .. \(.segment.max_key)"'

# ready-to-run SQL for the first difference, both sides
synq-recon run-drill suite.yaml --include orders -o json 2>/dev/null \
  | jq -r '.investigation_queries[0] | .source_query, .target_query'

Exit codes

Command	`0`	`1`	`2`
`run`, `run-check`, `run-drill`, `recheck`, `drill-deeper`	Everything matched	Differences found or the command failed	—
`run-remote --wait`, `trigger --wait`	Passed	A `--fail-on` condition was met	The run itself failed

Locally, 1 does not distinguish “found a difference” from “could not run”. If you need that distinction — in CI, say — either read -o json or use a backend run with --wait, whose codes are data-driven. See CI/CD and automation.

Audit logs

--audit-log writes the full record of a run: every query as executed, every count, checksum, timing, resolved variable, cutoff watermark and mismatch leaf.

synq-recon run suite.yaml --auto-drill --audit-log run.json      # a file
synq-recon run suite.yaml --auto-drill --audit-log audit-logs/   # a directory, auto-named

An audit log is what recheck and drill-deeper replay, and its shape is published as the audit-log schema. Some paths worth knowing:

jq -r '.reconciliations[].reconciliation.name' run.json
jq -r '.reconciliations[].stages[].quick_check_result? | select(.) | "\(.source_count) vs \(.target_count)"' run.json
jq -r '.reconciliations[].stages[].bisection_result?.mismatch_leaves[]?.segment | "\(.min_key)..\(.max_key)"' run.json

Two traps in that file, both of which return null rather than an error if you get them wrong:

Casing differs by source. A local --audit-log file is snake_case (bisection_result, mismatch_leaves). The same log fetched from the workspace with audit-logs get <invocation-id> -o json is camelCase (bisectionResult, mismatchLeaves).
Counts and checksums are strings. They are 64-bit integers, which the audit log encodes as quoted strings ("source_count": "10") — unlike run-check -o json, where they are numbers. Compare with tonumber.

What a run costs

Reconciliation runs real queries against real warehouses. On a consumption-priced platform those queries cost money, so it is worth knowing which commands spend before you put one in a loop.

Command	Warehouse cost
`check-config`	None. Never connects.
`check-config --db`	One planner call per query (`LIMIT 0`, no scan) plus a table-metadata sweep.
`run-check`	One aggregate query per side. Scans the compared columns once.
`run-drill`, `run --auto-drill`	Two queries per drill level. Cheap on mostly-identical data, expensive when most rows differ.
`recheck`, `drill-deeper`	The stage they replay, scoped to what the previous run left open — usually much less than starting over.
`suite`, `deployment`, `runs`, `audit-logs`, `upload-config`, `promote`	None. Workspace API only.

Where the engine reports them, a run prints its own statistics — including a rough BigQuery cost estimate:

Query Statistics
================
Bytes scanned: 4.2 GiB
Bytes billed:  4.2 GiB
Est. cost:     $0.03 (BigQuery on-demand @ $6.25/TiB billed)
Slot time:     1m12s
Rows read:     18,204,551

BigQuery and Snowflake report bytes; ClickHouse and PostgreSQL report planner rows; DuckDB, Athena and Databricks report nothing, so the block is simply absent. The cost figure is derived from a public list price that varies by region and edition — treat it as an order of magnitude, not an invoice.

Gate a large run before it happens

synq-recon run-check suite.yaml --max-table-bytes 50000000000   # 50 GB
synq-recon run-check suite.yaml --max-table-rows 1000000000

These make the run estimate each reconciliation’s scan before executing it — a dry run or EXPLAIN, not a scan — and warn when an estimate exceeds the threshold. Both default to 0 (off), so an automated run incurs no extra queries unless you ask for the gate. Platforms that advertise no estimate support simply produce no warnings, so a gate is never a false sense of security. It is a guard against the accidental full-table drill, not a hard limit.

Standalone and air-gapped runs

By default, a locally executed run reports its audit log to Coalesce Quality whenever it can resolve a credential, so local results appear in the workspace next to backend runs. It says so at INFO, naming the endpoint. This happens even when every database in the suite is local.

synq-recon run suite.yaml --auto-drill --no-report --audit-log run.json

--no-report — or RECON_NO_REPORT in the environment — resolves no credential and opens no connection to us at all. The terminal and the audit-log file are the only outputs.

This is a fully supported way to operate, not a debugging flag. Combined with the default privacy level, it means: only counts and checksums leave the database, and even those never leave the machine. For a regulated environment where a comparison must not cross a boundary, this is the mode to use — and the audit log is a self-contained, schema-validated record you can hand to a reviewer.

Reporting also needs the SCOPE_RECON_EDIT permission. A credential without it produces one warning naming the remedy; the comparison has already happened and its result is unaffected.

Local-only conveniences

Two features exist in the CLI and deliberately nowhere else:

Pre-flight table analysis. check-config --db reports size, keys, indexes and a suggested key column. See validate before you spend a query.
Environment overrides. --environment <name> / --env-file <path> apply a named set of overrides at load time — most usefully pointing a suite at a dev copy of a schema while you author against it.

Environments are local only. They are applied by the CLI as it loads a suite; the workspace does not apply them. A suite uploaded with upload-config and run with run-remote, or promoted, sees no overrides. Use them for local authoring, not to model dev and prod in the platform — that is what Development and Production are for.

Investigating results

Reading a segment tree, and following a finding to the rows.

Workspace suites

Save the suite, run it on our backend, put it on a schedule.

CLI reference

Every command and flag, generated from the binary.

Agent workflow

The same loop, written as an operating procedure for an agent.

​The three commands

run-check

run-drill

run --auto-drill

​Selecting and parameterising a run

​Reading the output

​Exit codes

​Audit logs

​What a run costs

​Gate a large run before it happens

​Standalone and air-gapped runs

​Local-only conveniences

​Next

Investigating results

Workspace suites

CLI reference

Agent workflow

The three commands

Selecting and parameterising a run

Reading the output

Exit codes

Audit logs

What a run costs

Gate a large run before it happens

Standalone and air-gapped runs

Local-only conveniences

Next