Agent Workflow

Read https://docs.synq.io/custom-integrations/agent-workflow and follow it. Write an integration for <your tool>. Its API is at <docs url>.

​

Your task

​

Ground rules

1. The entity id is a lowercase slug built from the source system’s stable id. Restrict it to A-Z a-z 0-9 : _ . -, shaped <tool>::<kind>::<source-id>. Every human-readable string — the title, accents, spaces — goes in name and description instead. See Entity ids.
2. Declare schemas before writing any lineage. A SELECT * expands only over an entity whose columns are known. Skip this and table-level edges still appear while column-level ones silently do not. Note that a declared ColumnLineage edge is not validated against the upstream schema — a typo in upstream_column is accepted and yields a broken hop, so verification in step 9 is what catches it.
3. Bind every name in SQL that is not a warehouse object. A SqlDefinition resolves names that live at a warehouse address. Anything else resolves to nothing and is dropped with no error. See Binding.
4. Never send category or governance_category on a check. They are resolved by the workspace’s own categorisation rules, and a producer value outranks those rules. Send package and kind only.
5. feature_id is a stable constant, never generated. A fresh id each run creates a new feature instead of replacing the previous one.
6. The entity group is sent whole, every run. A partial send is a deletion of everything omitted. If reading the source tool can fail halfway, skip the group write that run.
7. Read the lineage back at the end and fail loudly. Assert the edges you expect, not merely that some edge exists, and exit non-zero on a timeout or a non-OK CLL state. This is the only thing that distinguishes a working integration from a broken one. Keep it in production.
8. Set the gRPC authority when building the channel, or every RPC returns UNIMPLEMENTED. Step 0 below has the option for each language.

​

Step 0: setup

QUALITY_CLIENT_ID
QUALITY_CLIENT_SECRET
QUALITY_API_ENDPOINT    developer.synq.io (EU) | api.us.synq.io (US) | api.au.synq.io (AU)

• Go: grpc.WithAuthority(endpoint) as a dial option.
• Python: options=(("grpc.default_authority", endpoint),) on grpc.secure_channel.

​

Step 1: map the source tool

• Does it transform data? → trait is_model
• Is it a raw external input? → trait is_source
• Is it read by a person: a dashboard, a report? → trait is_bi_like
• Does it validate another object? → trait is_test_type
• Is it infrastructure nobody wants drawn? → trait is_ignored_in_lineage

​

Entity ids

1. From the source system’s stable id, never a display name. A title changes; a primary key does not. Changing an id creates a second entity rather than renaming the first.
2. From the alphabet A-Z a-z 0-9 : _ . - only. That is what the platform preserves verbatim. Use :: as the separator and prefix with the tool and object kind: <tool>::<kind>::<source-id>.
3. Lowercased. Ids are case-sensitive, so Metabase::Question::87 and metabase::question::87 are two entities and nothing warns you. Lowercase at the single place you build ids.

metabase::question::87          good
metabase::Revenue by category   bad — a title, a space, a capital

​

Naming entities you did not create

• The app URL <app host>/catalog/<entity_id>/overview — the segment after /catalog/ is the id. The app host is regional, matching the API endpoint above.
• The MCP server: search_entities and the other read tools.
• A webhook payload: each entity carries its identifier and an entity_url containing the id.
• BatchIdsByCoordinates, which maps a warehouse name to matching ids.

​

Resolve before you write

• A custom entity that does not exist is rejected at the write, so a misspelled custom id surfaces immediately.
• An entity on a connected platform is not checked; it binds on next ingestion. So a typo’d warehouse name and a not-yet-crawled table are indistinguishable in the lineage afterwards.

​

Use the read API as input

​

Step 2: types

• type_id is 1–1000, workspace-wide, with no allocator.
• Call ListTypes first and refuse to start if one of your numbers is taken under a different name. Two integrations picking the same band is otherwise silent corruption.
• name is a short caption shown on a badge, max 100 chars: "Kafka Topic", not a sentence.
• svg_icon is required, raw SVG bytes, max 1 MB.
• traits from step 1.

​

Step 3: entities

• id — CustomIdentifier, from the source system’s stable id.
• type_id — from step 2.
• name — as the tool shows it, max 500 chars. Do not shorten it to fit.
• description — Markdown, max 10 000 chars.
• annotations — the tool’s own filing system: collection, folder, workspace, object kind. Max 20 annotations, each with a name (max 50 chars) and up to 20 values (max 100 chars). These are what people filter the catalog by, so keep them to labels, not prose.

​

Step 4: schemas

​

Step 5: lineage

​

SqlDefinition

• dialect — one of SQL_DIALECT_{BIGQUERY,CLICKHOUSE,DATABRICKS,MYSQL,POSTGRESQL,REDSHIFT,SNOWFLAKE,DUCKDB,TRINO,MSSQL,ORACLE,ATHENA,FABRIC}.
• sql — as the tool actually executes it, max 1 000 000 chars.
• database_context — what unqualified names in the SQL resolve against.
• references — see below. Max 200.

​

Binding non-warehouse names

SqlTableReference {
  instance_name  ""                        empty parts default from database_context
  database_name  ""
  schema_name    ""
  object_name    "order_items_enriched"    the name exactly as the SQL writes it
  entity         <Identifier of the entity it means>
}

• Matching is case-insensitive.
• A binding beats a real warehouse table of the same name — use it as a deliberate override when you need one.
• Empty parts default from database_context, exactly as an unqualified name in the SQL does.
• Names that are warehouse objects need no binding. One definition can read a bound custom entity and a raw table in the same query, binding only the first.
• Bindings replace with the definition. One dropped from a later write is gone.

​

ColumnLineage

• It replaces whole. An edge missing from a write is withdrawn. Regenerate it from the tool’s metadata every run rather than diffing.
• A column edge declares the table edge it implies. Do not also write a Relationship.
• It wins over SqlDefinition. An entity with both is served from this declaration. So either declare only what the SQL cannot say, or make this the authority for the entity’s whole column lineage.
• An upstream that does not exist yet is remembered, and the edge appears once it does.
• state_at defaults to now; set it explicitly when backfilling.

​

Relationship

​

Code and Git

• code feature (feature_id per file or function, several allowed): the SQL, Python, YAML or JSON behind the entity, max 100 000 chars, with a code_type of CODE_TYPE_{SQL,PYTHON,JSON,YAML,DBT,SQLMESH,LOOKML,JAVASCRIPT,BASH,DAX,JAVA,SCALA,TEXT,MARKDOWN}.
• git_file_reference feature: repository URL (SSH preferred), branch, file path. With a code integration connected, Coalesce Quality reads the file’s history and shows recent commits against the entity when it fails.

​

Step 6: checks

1. ChecksRelationshipsService.UpsertCheckRelationships with check, checked and optional checked_columns.
2. check_category feature, feature_id: "check", carrying package (a grouping, max 50 chars) and kind (what the tool calls it, max 50 chars).

​

Step 7: the entity group

run 1:  group = A, B, C
run 2:  group = B, C, D   -> A is deleted

1. Read the current state: ListEntityFeatures (with entity_id unset for the whole workspace), ListRelationships, ListCheckRelationships.
2. Diff against the state you are about to write.
3. Withdraw what is gone, before the group write: DeleteEntityFeature, DeleteRelationships, DeleteCheckRelationships.
4. Then write the group, which removes the entities themselves.

​

Step 8: executions

• status — EXECUTION_STATUS_{OK,WARN,ERROR,CRITICAL}.
• created_at — required, must be in the past, rejected before 2022-01-01.
• started_at / finished_at — for long runs.
• extras.executed_sql — the SQL that ran.

​

Step 9: verify

• an expected node or edge missing from the response,
• a node whose cll_details.cll_state is neither OK nor unspecified — RESOLUTION_FAILED is usually a name you should have bound in step 5 and did not,
• the poll deadline expiring, or the read erroring.

​

Definition of done

• Ids are lowercase slugs from the source system’s stable ids, in A-Z a-z 0-9 : _ . -
• No title, accent, space or other human-readable text is in an id; it is in name
• ListTypes is called before claiming type numbers
• Every type declares its traits
• Every entity with columns declares its schema, before lineage is written
• Every non-warehouse name in a SqlDefinition has a references binding
• Every upstream is resolved with BatchResolveIdentifiers before lineage is written
• Checks send package and kind only
• All feature_id values are constants
• The group is sent whole, and skipped entirely on a partial source read
• Obsolete features, relationships and check relationships are explicitly deleted; the group write alone does not remove them
• Credentials are read from the environment
• The program polls GetLineage at the end and exits non-zero when it is empty
• Running the program twice produces no change on the second run

​

Reference

• Modelling your tool — the decisions in more depth
• Building an integration — the same steps with Go and Python code
• API surface — every method, scope and limit
• gRPC API reference — generated from the protos
• API scopes — every scope and the methods it unlocks
• Protos and SDKs
• Examples — runnable Go and Python