Lineage
Services
RelationshipsService
(synq.lineage.v1.RelationshipsService)
RelationshipsService allow management of relationships between entities. Relationships can be created, updated, and deleted between 2 custom entities, or between a custom entity and Synq native entity.enum There is no option to create relationships between 2 Synq native entities (dbt model, BI dashboard, etc.).
UpsertRelationships
UpsertRelationships (UpsertRelationshipsRequest) UpsertRelationshipsResponse
Create or update relationships between entities. If the relationship already exists, it will be updated, otherwise it will be created.
DeleteRelationships
DeleteRelationships (DeleteRelationshipsRequest) DeleteRelationshipsResponse
Delete relationships between entities. If the relationship does not exist, it will be ignored.
LineageService
(synq.lineage.v1.LineageService)
LineageService allows you to fetch:
- Entity level lineage from a starting point of one or more entities.
- Column Level lineage from a starting point of multiple columns of a single entity.
GetLineage
GetLineage (GetLineageRequest) GetLineageResponse
Messages
Relationship
Field | Type | Description |
---|---|---|
upstream | synq.entities.v1.Identifier | none |
downstream | synq.entities.v1.Identifier | none |
CllDetails
Field | Type | Description |
---|---|---|
columns | repeated Column | Column details for CLL. |
cll_state | CllState | State of the CLL parse. UNSPECIFIED if CLL was not requested. |
cll_messages | repeated string | Messages related to CLL. e.g. Description of parse errors, etc. |
Column
Column in a table-like asset (used in CLL mode).
Field | Type | Description |
---|---|---|
column_id | string | ID string for the column. This is the parsed column name. |
oneof _name.name | optional string | Original column name as fetched from the table. |
oneof _native_type.native_type | optional string | Column type as fetched from the table. |
ColumnDependency
Indicates data flow between columns. Source columns are used to compute value of target columns.
Field | Type | Description |
---|---|---|
source_node_idx | uint32 | Index of source node in the lineage nodes list. |
source_node_column_id | string | none |
target_node_idx | uint32 | Index of target node in the lineage nodes list. |
target_node_column_id | string | none |
Lineage
Lineage defines the lineage of table-like entities.
Field | Type | Description |
---|---|---|
nodes | repeated LineageNode | Nodes in the lineage with their identities and columns. |
node_dependencies | repeated NodeDependency | All edges in the lineage between nodes. This can be parsed to create a graph of all the nodes. |
is_cll | bool | Indicates whether the lineage was filtered for column level lineage (CLL). |
column_dependencies | repeated ColumnDependency | Dependencies between columns. Populated only for CLL. |
LineageNode
Node in a lineage graph representing one or more entiities (e.g. database table).
Field | Type | Description |
---|---|---|
ids | repeated synq.entities.v1.Identifier | All entities which have the same identity as this node. Must be at least one item. These are sorted by closeness to the type of the start point entities. e.g. if requesting lineage of a DBT source, first entity should be from DBT, similarly when viewing table it will be other tables. |
position | NodePosition | Position of the node in the lineage. |
oneof _cll_details.cll_details | optional CllDetails | Populated only for Column Level Lineage (CLL). |
NodeDependency
Indicates data flow between nodes. Source nodes are used to compute value of target nodes.
Field | Type | Description |
---|---|---|
source_node_idx | uint32 | Index of source node in the lineage nodes list. |
target_node_idx | uint32 | Index of target node in the lineage nodes list. |
DeleteRelationshipsRequest
Field | Type | Description |
---|---|---|
relationships | repeated Relationship | none |
DeleteRelationshipsResponse
UpsertRelationshipsRequest
Field | Type | Description |
---|---|---|
relationships | repeated Relationship | none |
UpsertRelationshipsResponse
EntitiesStartPoint
Field | Type | Description |
---|---|---|
entities | repeated synq.entities.v1.Identifier | none |
EntityColumnsStartPoint
Field | Type | Description |
---|---|---|
id | synq.entities.v1.Identifier | none |
column_names | repeated string | none |
GetLineageRequest
Field | Type | Description |
---|---|---|
lineage_direction | LineageDirection | none |
start_point | GetLineageStartPoint | none |
oneof _max_depth.max_depth | optional int32 | none |
GetLineageResponse
Field | Type | Description |
---|---|---|
lineage | Lineage | none |
GetLineageStartPoint
Possible starting points to get lineage from.
Field | Type | Description |
---|---|---|
oneof from.entities | EntitiesStartPoint | none |
oneof from.entity_columns | EntityColumnsStartPoint | none |