> ## Documentation Index
> Fetch the complete documentation index at: https://docs.synq.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Modelling Your Tool
> Choosing entity types, traits and the right lineage mechanism for each hop in your tool's graph
Before writing any code, four decisions. They are listed in order of how expensive they are
to get wrong.
## 1. Entity ids
Every entity is named by a `CustomIdentifier` — one string, unique within your workspace.
**Treat it as a slug, not a label.** It is a machine identifier: something a URL, a log line
and a config file can all carry unchanged. Everything a person reads goes in `name` and
`description`, which are separate fields and have no such restriction.
```
id metabase::question::87
name Revenue by product category — EMEA, Q3
```
The id never has to carry the title. The catalog shows `name`, which takes up to 500
characters of any UTF-8 you like: accents, punctuation, spaces, the tool's own wording. The
id only has to be stable and unique.
### Building the slug
Two rules, and a third worth adopting.
**Build it from the source system's stable id, never from a title.** Everything hangs off the
id: lineage edges, schemas, executions, check relationships, ownership, a saved view somebody
built. Changing an id does not rename an entity — it creates a second one and orphans the
first. A title changes; a primary key does not.
**Restrict it to `A-Z a-z 0-9 : _ . -`.** That is the alphabet the platform preserves
verbatim. `::` is its conventional separator, and `.`, `-` and `_` pass through untouched.
A prefix naming the tool and the object kind keeps two integrations in one workspace from
colliding, and keeps ids legible in a URL.
**Lowercase everything.** Ids are case-sensitive, so `Metabase::Question::87` and
`metabase::question::87` are two separate entities. Nothing warns you. Lowercasing at the one
place you build ids removes the whole class of bug, and costs nothing — the readable form
lives in `name` regardless.
```
metabase::question::87 good — stable id, safe alphabet, lowercase
metabase::Revenue by category bad — a title, a space, and a capital
```
### What happens if you ignore this
The string you send is not stored verbatim. Three things happen to it, in order:
1. It is truncated at **200 bytes**, with the tail replaced by a hash of the whole input so
two long ids sharing a prefix do not collapse into one.
2. Non-ASCII characters are replaced by their literal escape text, so `zamówienia` becomes
`zam_u00f3wienia`.
3. Every character outside `A-Z a-z 0-9 : _ . -` is folded to `_`.
**Two ids that differ only in folded characters become the same entity.** `metabase/question/87`
and `metabase question 87` both normalise to `metabase_question_87` — one entity, silently,
with the second write overwriting the first.
Slashes, spaces, `#`, `@`, `+` and `~` all fold. So does every accented character, into an
escape sequence nobody wants to read. Put that text in `name`, where it belongs and where it
survives intact.
The stored id is your slug prefixed with `custom-`, and that is what the API returns as
`entity_id`:
```
you send metabase::question::87
stored as custom-metabase::question::87
```
### Finding the entity\_id of an existing entity
Custom lineage points at entities that already exist — warehouse tables, dbt models,
dashboards.
**Always prefer the structured variant for the platform**: `bigquery_table`,
`snowflake_table`, `dbt_core_node` and the rest name an entity in the terms the platform
itself uses, which is what makes them readable, stable and checkable.
`synq_path` is the fallback. Reach for it in two cases: the platform has no structured
variant (Looker, Tableau, Coalesce Catalog and anything added since are addressable only this
way), or you already hold the entity's opaque `entity_id` and would rather pass it through
than take it apart. Four ways to get one:
* **From the app URL.** The catalog page for an entity is
`/catalog//overview`, where the app host is your region's
(`app.synq.io`, `app.us.synq.io`, `app.au.synq.io`). In
`https://app.synq.io/catalog/ch-nogwv291ou.europe_west4.gcp.clickhouse.cloud::default::runs/overview`
the id is `ch-nogwv291ou.europe_west4.gcp.clickhouse.cloud::default::runs`.
* **From the MCP server.** `search_entities` and the other read tools return entities with
their ids — see [Scout MCP](/scout/mcp).
* **From a webhook payload.** Every entity in an event carries its `identifier` and an
`entity_url` whose `/catalog/` segment is the id — see [Webhooks](/api-reference/webhook).
* **From the API.** Every identifier the API *returns* carries `entity_id`, and
`BatchIdsByCoordinates` turns a warehouse name into the ids that match it.
`entity_id` is output only. On a request it is ignored and the structured variant is what
resolves, so to name an entity *by* its opaque id, set the `synq_path` variant to that string.
### Resolve identifiers before you write lineage
`BatchResolveIdentifiers` (`synq.entities.resolve.v1`) turns any identifier — of any shape —
into its `entity_id`, and with `check_existence: true` tells you whether the entity is
actually there.
**The two kinds of upstream behave differently, and this is what tells them apart:**
* A **custom** entity that does not exist is **rejected** at the write. Nothing else would
report a misspelled custom id, so the API refuses rather than accepting a declaration that
produces nothing.
* An entity on a **connected platform** is not checked. It binds when that platform is next
ingested, so an edge written before the table is crawled is correct and is remembered.
That second case is the one to resolve up front: a not-yet-ingested table and a typo'd
warehouse name are indistinguishable from the lineage you get back.
It also answers a question the write API cannot. A single entity is reachable by several
identifiers — a dbt model and the warehouse table it builds are **one identity**, named two
ways. `BatchResolveIdentifiers` returns the whole identity group in `identities` and
`identity_synq_paths`, so you can tell that two identifiers you were treating as different
upstreams are the same entity, before writing two relationships that turn out to be one edge.
### The read API is input, not just verification
A custom integration does not only write. The public API's read surface is there to be driven
from your integration, and using it is usually the difference between a mapping that guesses
and one that resolves:
* **`BatchIdsByCoordinates`** — your tool reports a table as a string like
`analytics.order_items`. This maps warehouse coordinates to the entity ids that match, so
you can bind a name without hardcoding which warehouse it lives in.
* **`SearchEntities`** and **`ListEntities`** — find the entity your tool's metadata refers
to when it gives you a name rather than an address.
* **`GetSchema`** — read an upstream's real columns, so the column lineage you declare names
columns that exist rather than columns your tool believes exist.
* **`ListEntityTypeDefs`** and **`ListTypes`** — what types the workspace already has, which
is how you avoid claiming a type number another integration is using.
* **`Browse`** and **`GetFolderOf`** — where entities sit in the workspace's folder tree.
Treat these as the configuration input for the mapping, resolved at run time, rather than
building a static table of ids into your integration and watching it drift.
## 2. Entity types and their traits
A type is a kind of thing in your tool. It carries a short name, an SVG icon, and a set of
**traits** that every entity of the type inherits.
| Trait | Declares that entities of this type are | Effect |
| ----------------------- | ------------------------------------------- | ----------------------------------------------------------------------------------- |
| `is_model` | transformation models, the dbt-model analog | Canonical-entity ranking, monitorability, lineage treated as a transformation stage |
| `is_source` | raw external inputs to the graph | Ranked and rendered as a source |
| `is_bi_like` | dashboards, reports, other BI artifacts | Treated as a consumption endpoint |
| `is_test_type` | checks, pairing with declarative checks | Participates in coverage and check reporting |
| `is_ignored_in_lineage` | infrastructure you do not want drawn | Excluded from lineage rendering |
Traits are the difference between a catalog that understands your tool and one that merely
lists it. A type with no traits still appears everywhere, but it does not rank against dbt
models when Coalesce Quality picks a canonical entity, and it will not be offered where a
model is expected.
Types are numbered `1`–`1000` per workspace, and the number is the identity — reusing one
redefines the type. Keep a table of them in your integration's source.
## 3. Which lineage mechanism
This is the decision that repays thinking. Coalesce Quality offers four, and the right one
depends on what the hop actually has.
| You have | Use | Why |
| -------------------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------- |
| SQL, over real warehouse tables | `SqlDefinition` | Both table and column grain are derived from the SQL, and stay correct when the SQL changes |
| SQL, naming something the warehouse cannot address | `SqlDefinition` + `references` | The binding says what the name means; everything else is derived as usual |
| No SQL, but you know the column mapping | `ColumnLineage` | Declare the answer instead of deriving one |
| No SQL and no columns | `Relationship` | The edge is real; there is nothing to say at column grain |
| Something that validates another entity | `CheckCategory` + a check relationship | A check is not a stage data flows through |
**Reach for `SqlDefinition` first.** A declaration is a snapshot of what was true when it
was written; a parse follows the query. Every other mechanism is for what SQL cannot say.
### SqlDefinition
Give the SQL as the tool actually executes it, plus the dialect. Coalesce Quality parses it
and derives table-level and column-level lineage.
```
dialect: SQL_DIALECT_BIGQUERY
sql: SELECT customer_id, SUM(net_revenue) AS revenue
FROM analytics.order_items_enriched
GROUP BY 1
```
`database_context` supplies the instance, database and schema that unqualified names in the
SQL resolve against — the same defaults the warehouse itself would apply.
### SqlDefinition references — the binding
A `SqlDefinition` resolves names that live at a warehouse address. Anything else — another
custom entity, a queue, a service, an API, a saved query in a BI tool — has no address, so
`FROM order_items_enriched` matches nothing and is dropped.
**There is no error for this.** An empty result looks exactly like a query that genuinely
reads nothing, which is why it is the most common way a custom integration ships broken.
`SqlDefinition.references` binds the name, as the SQL writes it, to the entity it stands
for. Three consequences:
* **A binding beats the warehouse.** If a real table shared the name, the binding still
wins — which is what makes it usable as a deliberate override.
* **Empty parts default from `database_context`**, exactly as an unqualified name in the
SQL does. A binding of just `object_name: "orders_topic"`, under a definition whose
database is `kafka`, matches both `FROM kafka.orders_topic` and `FROM orders_topic`.
* **Names that *are* warehouse objects need no binding.** One definition can read a bound
custom entity and a raw warehouse table in the same query, binding only the first.
Matching is case-insensitive. The bindings are part of the definition and replace with it:
a binding dropped from a later write is gone, and the lineage it produced is withdrawn.
### ColumnLineage
For a component with no SQL for a parser to read — a transformation written in Python, a
hand-maintained mapping, a dashboard field assembled by the tool's own engine. You state
each edge: upstream entity, upstream column, this entity's column.
```
upstream = metabase::question::87, upstream_column = revenue, column = revenue
```
Three things to know:
* **It replaces whole.** An edge missing from a write is withdrawn, and deleting the
feature withdraws all of them. Only one `ColumnLineage` per entity, so re-writing it
under the same `feature_id` is how it is edited.
* **A column edge declares the table edge it implies.** The entity appears downstream of
each upstream named here whether or not a `Relationship` also says so.
* **It wins over `SqlDefinition`.** An entity with both is served from the declaration in
the SQL-lineage view — an explicit statement outranks a derived one. So declare here only
what the SQL cannot say, or state the entity's whole column lineage here and treat it as
the authority.
`state_at` defaults to now; set it explicitly when backfilling so an older declaration does
not overwrite a fresher one.
### Relationship
A plain upstream/downstream edge between two entities, with nothing said at column grain.
Use it where there genuinely are no columns — a subscription, an export job, a notification.
Relationships join a custom entity to another custom entity, or a custom entity to a native
one. **Two native entities cannot be joined** — their lineage comes from their own
integrations.
An edge is held between the two *entities* the endpoints resolve to, not between the
identifiers as written, and one entity accepts several spellings. The write response reports
the resolved pair and what the write did (`CREATED`, `UPDATED`, `DELETED`, `NOT_FOUND`) —
read it rather than assuming, because two relationships that resolve to the same pair are
one edge and only the last write survives.
### Checks
Something that validates another entity is not a stage data flows through, so it does not
get a lineage edge. Give the entity a `CheckCategory` feature and attach it with a **check
relationship**, optionally naming the columns it covers.
```
package: "metabase" grouping of check kinds
kind: "alert" what kind of check this is, in your tool's own words
```
**Write the check relationship before the `CheckCategory` feature.** Attaching a check
schedules no re-publish of its derived record, while writing a feature does — so doing it the
other way round leaves the check's stored state stale until the entity is written again.
**Leave `category` and `governance_category` unset.** They are resolved by your workspace's
own [categorisation rules](/analytics/check-categories), and a value sent by a producer
outranks those rules. Deriving one from your tool's alert kind — the obvious thing to do —
silently switches off every rule the workspace wrote, and nothing in the product explains
why. Send `package` and `kind`, which is what your tool actually says, and let the rules
decide.
## 4. Declare the schema, or lose the column grain silently
Declare every entity's columns before anything reads them. This is the step most easily
skipped and the most expensive to skip: a `SELECT *` over a bound entity expands only over an
entity whose columns are known. Skip it and the table-level edges still appear, with only the
column-level ones missing and nothing saying so.
**A declared `ColumnLineage` edge is not checked against the upstream's schema.** The read
joins on the column names the two sides state, so a typo in `upstream_column` is accepted and
produces a broken hop rather than an error — the edge exists, and the chain beyond it does
not. Declaring the schema does not protect you here; reading the lineage back is what catches
it.
Columns carry a name, a native type, a description, an ordinal position, and may nest for
struct and repeated types. Give ordinal positions for all columns or for none — when absent
they follow list order.
## Reading the graph back
Every way this goes wrong produces a valid-looking write and an empty graph. The two are
indistinguishable from the write side, so a custom integration ends by reading its own
lineage back and checking it is there. Lineage is computed asynchronously, so the read
polls.
This is worth keeping in the production integration, not just in the first version.
## Next
The walkthrough, in order, with the code
Every method, scope and limit