Skip to main content
Make these four decisions before you write any code. They are listed in order of how expensive they are to get wrong.

1. Entity ids

Every entity is named by a CustomIdentifier: one string, unique within your workspace. Treat it as a slug. It is a machine identifier, safe to use in URLs and config. Everything a person reads goes in name and description, which are separate fields and have no such restriction.
The id never has to carry the title. The catalog shows name, which takes up to 500 characters of any UTF-8 you like: accents, punctuation, spaces, the tool’s own wording. The id only has to be stable and unique.

Building the slug

Follow these three rules. Build it from the source system’s stable id, never from a title. Everything hangs off the id: lineage edges, schemas, executions, check relationships, ownership, a saved view somebody built. If you change an id, you create a second entity and orphan the first. A title changes; a primary key does not. Restrict it to A-Z a-z 0-9 : _ . -. That is the alphabet the platform preserves verbatim. :: is its conventional separator, and ., - and _ pass through untouched. A prefix naming the tool and the object kind keeps two integrations in one workspace from colliding, and keeps ids legible in a URL. Lowercase everything. Ids are case-sensitive, so Metabase::Question::87 and metabase::question::87 are two separate entities. Lowercase the id in the one place you build ids. The readable form goes in name.

What happens if you ignore this

The string you send is not stored verbatim. Three things happen to it, in order:
  1. It is truncated at 200 bytes, with the tail replaced by a hash of the whole input so two long ids sharing a prefix do not collapse into one.
  2. Non-ASCII characters are replaced by their literal escape text, so zamówienia becomes zam_u00f3wienia.
  3. Every character outside A-Z a-z 0-9 : _ . - is folded to _.
Two ids that differ only in folded characters become the same entity. metabase/question/87 and metabase question 87 both normalise to metabase_question_87. They are one entity, and the second write silently overwrites the first.Slashes, spaces, #, @, + and ~ all fold. Every accented character becomes an escape sequence. Put that text in name, where it is kept as written.
The stored id is your slug prefixed with custom-, and that is what the API returns as entity_id:

Finding the entity_id of an existing entity

Custom lineage points at entities that already exist: warehouse tables, dbt models, dashboards. Always prefer the structured variant for the platform: bigquery_table, snowflake_table, dbt_core_node and the rest name an entity in the terms the platform itself uses, which is what makes them readable, stable and checkable. synq_path is the fallback. Use it in two cases: the platform has no structured variant (Looker, Tableau, Coalesce Catalog and any other platform without one are addressable only this way), or you already hold the entity’s opaque entity_id and would rather pass it through than take it apart. Four ways to get one:
  • From the app URL. The catalog page for an entity is <app host>/catalog/<entity_id>/overview, where the app host is your region’s (app.synq.io, app.us.synq.io, app.au.synq.io). In https://app.synq.io/catalog/ch-nogwv291ou.europe_west4.gcp.clickhouse.cloud::default::runs/overview the id is ch-nogwv291ou.europe_west4.gcp.clickhouse.cloud::default::runs.
  • From the MCP server. search_entities and the other read tools return entities with their ids. See Scout MCP.
  • From a webhook payload. Every entity in an event carries its identifier and an entity_url whose /catalog/ segment is the id. See Webhooks.
  • From the API. Every identifier the API returns carries entity_id, and BatchIdsByCoordinates turns a warehouse name into the ids that match it.
entity_id is output only. On a request it is ignored and the structured variant is what resolves, so to name an entity by its opaque id, set the synq_path variant to that string.

Resolve identifiers before you write lineage

BatchResolveIdentifiers (synq.entities.resolve.v1) turns an identifier of any shape into its entity_id, and with check_existence: true tells you whether the entity is actually there. The two kinds of upstream behave differently, and this is what tells them apart:
  • A custom entity that does not exist is rejected at the write, so a misspelled custom id fails immediately.
  • An entity on a connected platform is not checked. It binds when that platform is next ingested, so an edge written before the table is crawled is correct and is remembered.
That second case is the one to resolve up front: a not-yet-ingested table and a typo’d warehouse name are indistinguishable from the lineage you get back. One entity can have several identifiers. A dbt model and the warehouse table it builds are one identity, named two ways. BatchResolveIdentifiers returns the whole identity group in identities and identity_synq_paths, so you can tell that two identifiers you were treating as different upstreams are the same entity, before writing two relationships that turn out to be one edge.

Use the read API as input

Your integration can also call the read API to resolve names at run time, so the mapping does not have to guess:
  • BatchIdsByCoordinates: your tool reports a table as a string like analytics.order_items. This maps warehouse coordinates to the entity ids that match, so you can bind a name without hardcoding which warehouse it lives in.
  • SearchEntities and ListEntities: find the entity your tool’s metadata refers to when it gives you a name rather than an address.
  • GetSchema: read an upstream’s real columns, so the column lineage you declare names columns that exist, not columns your tool believes exist.
  • ListEntityTypeDefs and ListTypes: what types the workspace already has, which is how you avoid claiming a type number another integration is using.
  • Browse and GetFolderOf: where entities sit in the workspace’s folder tree.
Resolve these at run time. A static table of ids in your integration drifts out of date.

2. Entity types and their traits

A type is a kind of thing in your tool. It carries a short name, an SVG icon, and a set of traits that every entity of the type inherits. A type with no traits still appears everywhere, but it does not rank against dbt models when Coalesce Quality picks a canonical entity, and it will not be offered where a model is expected. Types are numbered 1–1000 per workspace, and the number is the identity. Reusing a number redefines the type. Keep a table of them in your integration’s source.

3. Which lineage mechanism

Pick the mechanism for each hop based on what information that hop has. Reach for SqlDefinition first. A declaration is a snapshot of what was true when it was written; a parse follows the query. Every other mechanism is for what SQL cannot say.

SqlDefinition

Give the SQL as the tool actually executes it, plus the dialect. Coalesce Quality parses it and derives table-level and column-level lineage.
database_context supplies the instance, database and schema that unqualified names in the SQL resolve against. These are the same defaults the warehouse itself would apply.

SqlDefinition references (the binding)

A SqlDefinition resolves names that live at a warehouse address. Anything else (another custom entity, a queue, a service, an API, a saved query in a BI tool) has no address, so FROM order_items_enriched matches nothing and is dropped. There is no error for this. An empty result looks exactly like a query that genuinely reads nothing, which is why it is the most common way a custom integration ships broken. SqlDefinition.references binds the name, as the SQL writes it, to the entity it stands for. Three consequences:
  • A binding beats the warehouse. If a real table has the same name, the binding still wins. You can use this as a deliberate override.
  • Empty parts default from database_context, exactly as an unqualified name in the SQL does. A binding of just object_name: "orders_topic", under a definition whose database is kafka, matches both FROM kafka.orders_topic and FROM orders_topic.
  • Names that are warehouse objects need no binding. One definition can read a bound custom entity and a raw warehouse table in the same query, binding only the first.
Matching is case-insensitive. The bindings are part of the definition and replace with it: a binding dropped from a later write is gone, and the lineage it produced is withdrawn.

A copy between two tables you already have

A replication tool, a CDC pipeline or your own loader copies a table into another database or warehouse. Both tables are already in Coalesce Quality; the hop between them is not. Model the copy itself as a custom entity, one per copied table. It is what fails, lags or drops a column, so it is where an incident belongs. One entity for a whole connection would make every source table upstream of every target table it writes. Give that entity a SqlDefinition describing the copy, and bind both table names:
  • The SQL is parsed, never run. source_table and target_table are placeholders that only have to match the references. A bound name resolves straight to its entity, on whichever platform that is, so the source and the target can be in different warehouses. Use the source side’s dialect.
  • Write the column list on the INSERT. It pairs each selected column with its target column, renames included. The lineage runs source column → copy → target column.
  • No relationships are needed. The same definition gives the table-level edges too.
replication_lineage is a runnable example.

ColumnLineage

For a component with no SQL for a parser to read: a transformation written in Python, a hand-maintained mapping, a dashboard field assembled by the tool’s own engine. You state each edge: upstream entity, upstream column, this entity’s column.
Three things to know:
  • It replaces whole. An edge missing from a write is withdrawn, and deleting the feature withdraws all of them. Only one ColumnLineage per entity, so re-writing it under the same feature_id is how it is edited.
  • A column edge declares the table edge it implies. The entity appears downstream of each upstream named here whether or not a Relationship also says so.
  • It wins over SqlDefinition. An entity with both is served from the declaration in the SQL-lineage view. An explicit statement outranks a derived one. So declare here only what the SQL cannot say, or state the entity’s whole column lineage here and treat it as the authority.
state_at defaults to now; set it explicitly when backfilling so an older declaration does not overwrite a fresher one.

Relationship

A plain upstream/downstream edge between two entities, with nothing said at column grain. Use it where there are no columns: a subscription, an export job, a notification. Relationships join a custom entity to another custom entity, or a custom entity to a native one. Two native entities cannot be joined. Their lineage comes from their own integrations. When your tool is what moves data between two of them, model it as described in A copy between two tables you already have. An edge is held between the two entities the endpoints resolve to, not between the identifiers as written, and one entity accepts several spellings. The write response reports the resolved pair and what the write did (CREATED, UPDATED, DELETED, NOT_FOUND). Read it, because two relationships that resolve to the same pair are one edge and only the last write survives.

Checks

Something that validates another entity is not a stage data flows through, so it does not get a lineage edge. Give the entity a CheckCategory feature and attach it with a check relationship, optionally naming the columns it covers.
Write the check relationship first, then the CheckCategory feature. Leave category and governance_category unset. They are resolved by your workspace’s own categorisation rules, and a value sent by a producer outranks those rules. If you derive one from your tool’s alert kind, the workspace’s rules do not apply to those checks. Send package and kind, which is what your tool actually says, and let the rules decide.

4. Declare the schema, or lose the column grain silently

Declare every entity’s columns before anything reads them. A SELECT * over a bound entity expands only over an entity whose columns are known. Skip it and the table-level edges still appear, with only the column-level ones missing and nothing saying so.
A declared ColumnLineage edge is not checked against the upstream’s schema. A typo in upstream_column gives a broken hop and no error. Read the lineage back to catch it.
Columns carry a name, a native type, a description, an ordinal position, and may nest for struct and repeated types. Give ordinal positions for all columns or for none. When absent, they follow list order.

Reading the graph back

Every way this goes wrong produces a valid-looking write and an empty graph. The two are indistinguishable from the write side, so a custom integration ends by reading its own lineage back and checking it is there. Lineage is computed asynchronously, so the read polls. Keep this check in the production integration.

Next

Building an integration

The walkthrough, in order, with the code

API surface

Every method, scope and limit