IngestLineage
IngestLineage pushes data-flow lineage between warehouse objects (and,
optionally, between their columns) into the platform. Send the lineage you
extract from the warehouse — for example from Snowflake’s GET_LINEAGE — as a
batch of directed edges. Use replace_mode to choose between an
incremental update (MERGE) and an authoritative refresh of a region of the
warehouse (REPLACE_SCOPE); REPLACE_SCOPE is what lets the platform expire
edges that no longer exist. Safe to retry: re-sending the same batch
converges to the same state.
Authorizations
Bearer authentication header of the form Bearer <token>, where <token> is your auth token.
Path Parameters
Identifier of the warehouse connection this lineage was read from. Use the same connection_id you use for the other Ingest* calls of this warehouse so the lineage attaches to the right objects.
A unique id for this batch (UUID). Re-sending a batch with the same upload_id is de-duplicated, so it is safe to retry.
Body
IngestLineageRequest carries one batch of lineage edges for a single
warehouse connection. Send many batches with the same connection_id over
time; each batch is interpreted against the others according to
replace_mode.
Time at which the lineage was extracted from the warehouse. timestamp.gt = 2022-01-01T00:00:00Z timestamp.gt_lt = 2022-01-01T00:00:00Z timestamp.gt_lt_exclusive = 2022-01-01T00:00:00Z timestamp.gt_lte = 2022-01-01T00:00:00Z timestamp.gt_lte_exclusive = 2022-01-01T00:00:00Z timestamp.lt_now = true
"2023-01-15T01:30:15.01Z"
"2024-12-25T12:00:00Z"
How the lineage was derived. Drives the precedence of the resulting edges and keeps lineage of different origins from overwriting one another.
LINEAGE_SOURCE_UNSPECIFIED, LINEAGE_SOURCE_DWH_NATIVE, LINEAGE_SOURCE_QUERY_HISTORY, LINEAGE_SOURCE_DECLARED, LINEAGE_SOURCE_VIEW_DEFINITION How this batch supersedes previously-ingested lineage. Defaults to MERGE (incremental) when unset.
LINEAGE_REPLACE_MODE_UNSPECIFIED, LINEAGE_REPLACE_MODE_MERGE, LINEAGE_REPLACE_MODE_REPLACE_SCOPE Required when replace_mode is REPLACE_SCOPE; ignored otherwise.
- databases
- schemas
- whole_connection
The lineage edges in this batch. Empty in REPLACE_SCOPE mode is a valid way to expire all lineage inside the scope.
Response
Success
IngestLineageResponse is intentionally empty; a successful call means the batch was accepted for processing.