Agent workflow

​

1. The loop

Move on when it prints Configuration check passed!. Fix errors here rather than discovering them mid-run. Warnings about NOW() / CURRENT_DATE are real — see Never do these.

Move on when every connection is OK and every query validates. A failure here is a wrong table name, a missing grant, or a column that does not exist — all of which would otherwise surface as a confusing mid-run error. --db does not run the suite’s setup: SQL, so a suite whose tables are created by setup will report every query as failed. That is expected; skip to step 3 for such suites.

If it reports MATCH, the datasets agree and you are done. If it reports MISMATCH, go to step 4. Exit code 1 means “differences found”, which is a result, not a failure.

Move on when the drill reports mismatch leaves with usable key ranges. If it reports thousands of leaves, stop and read Never do these — you are drilling something that should be localised with an aggregate comparison first.

Run auth whoami first. Move on when it prints the suite id.

Move on when the run completes. --wait exits 0 if everything passed, 1 if a --fail-on condition was hit, 2 if execution itself failed. Pass --drill=false explicitly if you do not want a drill — omitting the flag does not mean “off”.

A deployment is a snapshot, not a pointer: editing the suite afterwards changes nothing in production until you re-promote. Re-promoting preserves settings you omit; a fresh promote applies defaults, and the drill default on a fresh promote is on.

​

2. Never do these

variables:
  start_date: "{{ today - 7d }}"
  end_date: "{{ today }}"

​

3. Confirm the target before any mutation

synq-recon auth whoami

1. Client credentials — --client-id / --client-secret, or QUALITY_CLIENT_ID / QUALITY_CLIENT_SECRET, or the synq: block in the suite.
2. An API token — QUALITY_TOKEN.
3. The browser login cached by synq-recon auth login, refreshed automatically.

QUALITY_REGION > the last login > eu.

​

4. Authoring a suite

name: orders-consistency
title: "Orders consistency"

connections:
  gateway:
    postgres:
      host: db.internal
      port: 5432
      database: payments
      username: ${PG_USER}
      password: ${PG_PASSWORD}
  warehouse:
    snowflake:
      account: myorg.us-east-1
      warehouse: COMPUTE_WH
      role: ANALYST
      username: ${SF_USER}
      password: ${SF_PASSWORD}
      databases: [ANALYTICS]

reconciliations:
  orders-daily:
    source:
      connection: gateway
      table: public.orders
    target:
      connection: warehouse
      table: ANALYTICS.PUBLIC.ORDERS
    key_columns: [order_id]
    mode: row_checksum

• table: beats query: when you can use it. With table: the tool resolves the columns for you and you can narrow with columns:, exclude_columns: and where:. Reach for query: only when the comparison genuinely needs SQL.
• Key columns drive everything. Bisection orders and range-filters on them, so they must be unique and, ideally, indexed. check-config --db suggests one and warns if yours is not indexed.
• Modes. row_count compares counts only (fastest, misses modified values); row_checksum compares counts and a checksum of every column (the default, catches any difference); aggregate compares grouped measures like SUM and COUNT with a tolerance, and drills down through a group_columns hierarchy. full is an accepted legacy spelling of row_checksum; write row_checksum.
• Column names differing only in case match automatically. For genuinely different names use column_mapping.
• Keep credentials out of the suite. Put them in a git-ignored .connections.yaml (auto-discovered, or passed with --connections) keyed by the same connection names. This is also what lets the same suite run locally and in the workspace, where the names bind to integrations instead.

• Configuration reference — every field, type, constraint and default, rendered.
• config.schema.json — the machine-readable form. Point your editor at it for completion and validation:

# yaml-language-server: $schema=https://schemas.synq.io/synq-recon/v1/config.schema.json

​

Avoiding false differences

​

5. Reading the output

# names that differ
synq-recon run-check suite.yaml -o json | jq -r '.results[] | select(.match == false) | .name'

# key ranges where the difference lives
synq-recon run-drill suite.yaml --include orders-daily -o json \
  | jq -r '.mismatch_leaves[] | "\(.segment.min_key) .. \(.segment.max_key)"'

# ready-to-run SQL for one difference, both sides
synq-recon run-drill suite.yaml --include orders-daily -o json \
  | jq -r '.investigation_queries[0] | .source_query, .target_query'

jq -r '.reconciliations[].reconciliation.name' audit.json
jq -r '.reconciliations[].stages[].quick_check_result? | select(.) | "\(.source_count) vs \(.target_count)"' audit.json
jq -r '.reconciliations[].stages[].bisection_result?.mismatch_leaves[]?.segment | "\(.min_key)..\(.max_key)"' audit.json

Casing. The local --audit-log file is snake_case (bisection_result, mismatch_leaves). The same log fetched from the workspace with audit-logs get <invocation-id> -o json is camelCase (bisectionResult, mismatchLeaves). A jq path written for one source silently returns null against the other.

Counts are strings. Row counts and checksums are 64-bit integers, which JSON encodes as quoted strings in the audit log ("source_count": "10") — unlike run-check -o json, where they are numbers. Compare with tonumber.

​

6. What each command costs

​

7. Investigating a finished run

# Did the difference go away? (e.g. after a backfill)
synq-recon recheck audit.json --connections .connections.yaml

# Narrow every still-open segment to 100-row ranges
synq-recon drill-deeper audit.json --connections .connections.yaml --threshold 100

# Break each divergent aggregate group down by another column
synq-recon drill-deeper audit.json --add-group-column region

• Queries are replayed, not re-derived. The audit log holds each query after variable interpolation and cutoff resolution, so a re-check compares the same window the original run did — which is what makes “did the gap move?” a meaningful question. It also means a replay will not pick up an edited query or variable until you pass --reresolve. --reresolve is required for an as_of snapshot, which can otherwise never turn green.
• Only what was left open resumes. drill-deeper picks up the segments the previous run stopped at on its row threshold or depth limit; --depth counts from the depth they already reached, so it means “this many more levels”. Segments that could not be split further, and aggregate groups present on only one side, are reported and skipped rather than silently dropped. An aggregate drill already visits every configured group column, so resuming one needs at least one --add-group-column.
• recheck re-runs only the reconciliations that did not pass. --all re-runs every one.
• Credentials never come from an audit log — it records connection names only. Pass --connections, exactly as for a normal run.

​

8. Command reference

​

Local

​

Workspace

​

Flags accepted by every command

​

9. Worked example

cd synq-recon
go build -o synq-recon ./cmd/synq-recon      # needs CGO for DuckDB

./synq-recon check-config examples/ecommerce.yaml
# Configuration check passed!

./synq-recon run-check examples/ecommerce.yaml --include ecommerce-orders --no-report

Reconciliation: E-Commerce Orders (Payment Gateway vs Warehouse) (ecommerce-orders)
Mode: row_checksum

             Source              Target
Row Count    10                  9

Status: MISMATCH
  - Source has 1 more rows than target

Hint: Run 'synq-recon run-drill <config> --include ecommerce-orders' to locate differences.

./synq-recon run examples/ecommerce.yaml --include ecommerce-orders \
  --auto-drill --no-report --audit-log /tmp/orders.json

Depth 0: Segment [*, *)
  Source: 10 rows, checksum: -139824023944409
  Target: 9 rows, checksum: -144133400916972
  Status: MISMATCH -> drilling down (2/10 rows in mismatched segments)
  Depth 1: Segment [1,005, 1,006)
    Source: 1 rows, checksum: 53805022444628
    Target: 0 rows, checksum: 0
    Status: MISMATCH (leaf: threshold_reached)
  Depth 1: Segment [1,010, 1,011)
    Source: 1 rows, checksum: 20466544489071
    Target: 1 rows, checksum: 69962189961136
    Status: MISMATCH (leaf: threshold_reached)

./synq-recon run-drill examples/ecommerce.yaml --include ecommerce-orders \
  --no-report -o json 2>/dev/null \
  | jq -r '.investigation_queries[] | .source_query, .target_query, "--"'

printf 'connections:\n  test-db:\n    duckdb:\n      database: ":memory:"\n' > /tmp/conn.yaml
./synq-recon recheck /tmp/orders.json --connections /tmp/conn.yaml --no-report

RECONCILIATION    BEFORE       AFTER        CHANGE
----------------  -----------  -----------  ------------------------------------------
ecommerce-orders  mismatched   mismatched   still mismatched (row gap unchanged at -1)

./synq-recon run examples/ecommerce.yaml --include inventory-reconciliation \
  --auto-drill --no-report

​

10. When something goes wrong