Services

EntityIncidentsService

(synq.status.v1.EntityIncidentsService)

EntityIncidentsService is the service which retrieves entity status.

GetIncidents

GetIncidents (GetIncidentsRequest) GetIncidentsResponse

Get specific entity status by its identifier.

BatchGetIncidents

BatchGetIncidents (BatchGetIncidentsRequest) BatchGetIncidentsResponse

Get multiple entity statuses by their identifiers.

EntityIssuesService

(synq.status.v1.EntityIssuesService)

EntityIssuesService is the service which retrieves entity issues status.

GetIssuesStatus

GetIssuesStatus (GetIssuesStatusRequest) GetIssuesStatusResponse

Get specific entity status by its identifier.

BatchGetIssuesStatus

BatchGetIssuesStatus (BatchGetIssuesStatusRequest) BatchGetIssuesStatusResponse

Get multiple entity statuses by their identifiers.

RelationshipsService

(synq.lineage.v1.RelationshipsService)

RelationshipsService allow management of relationships between entities. Relationships can be created, updated, and deleted between 2 custom entities, or between a custom entity and Synq native entity.enum There is no option to create relationships between 2 Synq native entities (dbt model, BI dashboard, etc.).

UpsertRelationships

UpsertRelationships (UpsertRelationshipsRequest) UpsertRelationshipsResponse

Create or update relationships between entities. If the relationship already exists, it will be updated, otherwise it will be created.

DeleteRelationships

DeleteRelationships (DeleteRelationshipsRequest) DeleteRelationshipsResponse

Delete relationships between entities. If the relationship does not exist, it will be ignored.

LineageService

(synq.lineage.v1.LineageService)

LineageService allows you to fetch:

  • Entity level lineage from a starting point of one or more entities.
  • Column Level lineage from a starting point of multiple columns of a single entity.

GetLineage

GetLineage (GetLineageRequest) GetLineageResponse

EntitiesService

(synq.entities.v1.EntitiesService)

EntitiesService is a service for managing custom entities. Entities can represent various data platform concepts such as services, consumers, applications or data pipelines that are not natively available in Synq.

Entities are identified by a unique identifier and can be created, updated, read and deleted.

GetEntity

GetEntity (GetEntityRequest) GetEntityResponse

Get specific entity by its identifier.

BatchGetEntities

BatchGetEntities (BatchGetEntitiesRequest) BatchGetEntitiesResponse

Get multiple entities by their identifiers.

UpsertEntity

UpsertEntity (UpsertEntityRequest) UpsertEntityResponse

Create or update an entity. If the entity does not exist, it will be created, otherwise it will be updated. Entities are identified and deduplicated by their Identifier in a scope of a given customer workspace.

DeleteEntity

DeleteEntity (DeleteEntityRequest) DeleteEntityResponse

Delete an entity by its identifier. If the entity does not exist, it will be a no-op.

Messages

ScopeAuthorization

(synq.v1.ScopeAuthorization)

FieldTypeDescription
scopesrepeated Scopenone

EntityIncident

(synq.status.v1.EntityIncident)

FieldTypeDescription
id stringUnique identifier of the incident.
url stringURL of the incident.
name stringName of the incident give by the user.
created_by stringInformation about author of the incident.
created_at TimestampTime when the incident was created.
updated_by stringInformation about the user who last updated the incident.
updated_at TimestampTime when the incident was last updated.
is_resolved boolIndicates if all issues inside the incident are resolved.

BatchGetIncidentsRequest

(synq.status.v1.BatchGetIncidentsRequest)

BatchGetIncidentsRequest is the request message for the BatchGetIssues method.

FieldTypeDescription
requestsrepeated GetIncidentsRequestnone

BatchGetIncidentsResponse

(synq.status.v1.BatchGetIncidentsResponse)

BatchGetIncidentsResponse is the response message for the BatchGetIssues method.

FieldTypeDescription
responsesrepeated GetIncidentsResponsenone

GetIncidentsRequest

(synq.status.v1.GetIncidentsRequest)

GetIncidentsRequest is the request message for the GetEntity method.

FieldTypeDescription
id IdentifierIdentifier of the entity to get incidents for.
fetch_upstream_incidents boolRetrieve incidents of upstream entities.

GetIncidentsResponse

(synq.status.v1.GetIncidentsResponse)

GetIncidentsResponse is the response message for the GetEntity method.

FieldTypeDescription
id IdentifierIdentifier of the entity.
entity_open_incidentsrepeated EntityIncidentOpen incident associated with the entity requested.
upstream_open_incidentsrepeated EntityIncidentOpen incidents associated with the upstream entities if requested.

BatchGetIssuesStatusRequest

(synq.status.v1.BatchGetIssuesStatusRequest)

BatchGetIssuesStatusRequest is the request message for the BatchGetIssues method.

FieldTypeDescription
requestsrepeated GetIssuesStatusRequestnone

BatchGetIssuesStatusResponse

(synq.status.v1.BatchGetIssuesStatusResponse)

BatchGetIssuesStatusResponse is the response message for the BatchGetIssues method.

FieldTypeDescription
responsesrepeated GetIssuesStatusResponsenone

GetIssuesStatusRequest

(synq.status.v1.GetIssuesStatusRequest)

GetIssuesStatusRequest is the request message for the GetEntity method.

FieldTypeDescription
id IdentifierIdentifier of the entity to get status for.
fetch_upstream_status boolRetrieve status of upstream entities.

GetIssuesStatusResponse

(synq.status.v1.GetIssuesStatusResponse)

GetIssuesStatusResponse is the response message for the GetEntity method.

FieldTypeDescription
id IdentifierIdentifier of the entity.
entity_issues_status IssueStatusStatus of the entity requested.
entity_issues_summary IssuesSummarynone
upstream_issues_status IssueStatusOptionally, the status of all upstream entities.
upstream_issues_summary IssuesSummarynone

GetIssuesStatusResponse.IssuesSummary

(synq.status.v1.GetIssuesStatusResponse.IssuesSummary)

FieldTypeDescription
total_count int32Number of issues for the entity.
warn_count int32Number of issues for the entity with status ‘warn’.
error_count int32Number of issues for the entity with status ‘error’.
critical_count int32Number of issues for the entity with status ‘critical’.

Relationship

(synq.lineage.v1.Relationship)

FieldTypeDescription
upstream Identifiernone
downstream Identifiernone

CllDetails

(synq.lineage.v1.CllDetails)

FieldTypeDescription
columnsrepeated ColumnColumn details for CLL.
cll_state CllStateState of the CLL parse. UNSPECIFIED if CLL was not requested.
cll_messagesrepeated stringMessages related to CLL. e.g. Description of parse errors, etc.

Column

(synq.lineage.v1.Column)

Column in a table-like asset (used in CLL mode).

FieldTypeDescription
column_id stringID string for the column. This is the parsed column name.
oneof _name.nameoptional stringOriginal column name as fetched from the table.
oneof _native_type.native_typeoptional stringColumn type as fetched from the table.

ColumnDependency

(synq.lineage.v1.ColumnDependency)

Indicates data flow between columns. Source columns are used to compute value of target columns.

FieldTypeDescription
source_node_idx uint32Index of source node in the lineage nodes list.
source_node_column_id stringnone
target_node_idx uint32Index of target node in the lineage nodes list.
target_node_column_id stringnone

Lineage

(synq.lineage.v1.Lineage)

Lineage defines the lineage of table-like entities.

FieldTypeDescription
nodesrepeated LineageNodeNodes in the lineage with their identities and columns.
node_dependenciesrepeated NodeDependencyAll edges in the lineage between nodes. This can be parsed to create a graph of all the nodes.
is_cll boolIndicates whether the lineage was filtered for column level lineage (CLL).
column_dependenciesrepeated ColumnDependencyDependencies between columns. Populated only for CLL.

LineageNode

(synq.lineage.v1.LineageNode)

Node in a lineage graph representing one or more entities (e.g. database table).

FieldTypeDescription
idsrepeated IdentifierAll entities which have the same identity as this node. Must be at least one item. These are sorted by closeness to the type of the start point entities. e.g. if requesting lineage of a DBT source, first entity should be from DBT, similarly when viewing table it will be other tables.
position NodePositionPosition of the node in the lineage.
oneof _cll_details.cll_detailsoptional CllDetailsPopulated only for Column Level Lineage (CLL).

NodeDependency

(synq.lineage.v1.NodeDependency)

Indicates data flow between nodes. Source nodes are used to compute value of target nodes.

FieldTypeDescription
source_node_idx uint32Index of source node in the lineage nodes list.
target_node_idx uint32Index of target node in the lineage nodes list.

DeleteRelationshipsRequest

(synq.lineage.v1.DeleteRelationshipsRequest)

FieldTypeDescription
relationshipsrepeated Relationshipnone

DeleteRelationshipsResponse

(synq.lineage.v1.DeleteRelationshipsResponse)

UpsertRelationshipsRequest

(synq.lineage.v1.UpsertRelationshipsRequest)

FieldTypeDescription
relationshipsrepeated Relationshipnone

UpsertRelationshipsResponse

(synq.lineage.v1.UpsertRelationshipsResponse)

EntitiesStartPoint

(synq.lineage.v1.EntitiesStartPoint)

FieldTypeDescription
entitiesrepeated Identifiernone

EntityColumnsStartPoint

(synq.lineage.v1.EntityColumnsStartPoint)

FieldTypeDescription
id Identifiernone
column_namesrepeated stringnone

GetLineageRequest

(synq.lineage.v1.GetLineageRequest)

FieldTypeDescription
lineage_direction LineageDirectionnone
start_point GetLineageStartPointnone
oneof _max_depth.max_depthoptional int32none

GetLineageResponse

(synq.lineage.v1.GetLineageResponse)

FieldTypeDescription
lineage Lineagenone

GetLineageStartPoint

(synq.lineage.v1.GetLineageStartPoint)

Possible starting points to get lineage from.

FieldTypeDescription
oneof from.entities EntitiesStartPointnone
oneof from.entity_columns EntityColumnsStartPointnone

BatchGetEntitiesRequest

(synq.entities.v1.BatchGetEntitiesRequest)

BatchGetEntitiesRequest is the request message for the BatchGetEntities method.

FieldTypeDescription
idsrepeated IdentifierIdentifiers of the entities to get.

BatchGetEntitiesResponse

(synq.entities.v1.BatchGetEntitiesResponse)

BatchGetEntitiesResponse is the response message for the BatchGetEntities method.

FieldTypeDescription
entitiesrepeated EntityThe entities that were retrieved.

DeleteEntityRequest

(synq.entities.v1.DeleteEntityRequest)

DeleteEntityRequest is the request message for the DeleteEntity method.

FieldTypeDescription
id IdentifierIdentifier of the entity to delete.

DeleteEntityResponse

(synq.entities.v1.DeleteEntityResponse)

DeleteEntityResponse is the response message for the DeleteEntity method.

GetEntityRequest

(synq.entities.v1.GetEntityRequest)

GetEntityRequest is the request message for the GetEntity method.

FieldTypeDescription
id IdentifierIdentifier of the entity to get.

GetEntityResponse

(synq.entities.v1.GetEntityResponse)

GetEntityResponse is the response message for the GetEntity method.

FieldTypeDescription
entity EntityThe entity that was retrieved.

UpsertEntityRequest

(synq.entities.v1.UpsertEntityRequest)

UpsertEntityRequest is the request message for the UpsertEntity method.

FieldTypeDescription
entity EntityThe entity to create or update.

UpsertEntityResponse

(synq.entities.v1.UpsertEntityResponse)

UpsertEntityResponse is the response message for the UpsertEntity method.

AirflowDagIdentifier

(synq.entities.v1.AirflowDagIdentifier)

FieldTypeDescription
integration_id stringSynq integration_id that identifies the Airflow instance
dag_id stringAirflow dag_id that identifies the DAG

AirflowTaskIdentifier

(synq.entities.v1.AirflowTaskIdentifier)

FieldTypeDescription
integration_id stringSynq integration_id that identifies the Airflow instance
dag_id stringAirflow dag_id that identifies the DAG
task_id stringAirflow task_id that identifies the task within the DAG

BigqueryTableIdentifier

(synq.entities.v1.BigqueryTableIdentifier)

FieldTypeDescription
project stringBigQuery project
dataset stringBigQuery dataset id
table stringBigQuery table name

ClickhouseTableIdentifier

(synq.entities.v1.ClickhouseTableIdentifier)

FieldTypeDescription
host stringClickhouse host inclusive of port
schema stringClickhouse database
table stringClickhouse table

CustomIdentifier

(synq.entities.v1.CustomIdentifier)

FieldTypeDescription
id stringId that identifies the custom entity The Id should be unique within the custom entity Identifier.

DatabricksTableIdentifier

(synq.entities.v1.DatabricksTableIdentifier)

FieldTypeDescription
workspace stringnone
catalog stringDatabricks catalog
schema stringDatabricks schema
table stringDatabricks table or view

DataproductIdentifier

(synq.entities.v1.DataproductIdentifier)

FieldTypeDescription
id stringDataproduct id that identifies the Dataproduct

DbtCloudNodeIdentifier

(synq.entities.v1.DbtCloudNodeIdentifier)

FieldTypeDescription
project_id stringYour dbt Cloud project id
account_id stringYour dbt Cloud account id
node_id stringDbt node_id that identifies one of dbt DAG nodes (model, test, etc)

DbtCoreNodeIdentifier

(synq.entities.v1.DbtCoreNodeIdentifier)

FieldTypeDescription
integration_id stringSynq integration_id that identifies the dbt Core project
node_id stringDbt node_id that identifies one of dbt DAG nodes (model, test, etc)

Identifier

(synq.entities.v1.Identifier)

Identifier is a unique reference to an entity in Synq system. Entity identifiers are designed to closely mimic identifiers used by data platforms and tools. To construct an identifier, you need to know the kind of the entity and the ids that you would normally use to identify it in the data platform or tool. For example, to identify a table in BigQuery, you would need to know the project, dataset, and table names.

FieldTypeDescription
oneof id.dbt_core_node DbtCoreNodeIdentifierDbt node that identifies one of dbt DAG nodes (model, test, etc) in dbt Core project
oneof id.dbt_cloud_node DbtCloudNodeIdentifierDbt node that identifies one of dbt DAG nodes (model, test, etc) in dbt Cloud project
oneof id.bigquery_table BigqueryTableIdentifierBigQuery table identifier
oneof id.snowflake_table SnowflakeTableIdentifierSnowflake table identifier
oneof id.redshift_table RedshiftTableIdentifierRedshift table identifier
oneof id.postgres_table PostgresTableIdentifierPostgres table identifier
oneof id.mysql_table MysqlTableIdentifierMysql table identifier
oneof id.clickhouse_table ClickhouseTableIdentifierClickhouse table identifier
oneof id.databricks_table DatabricksTableIdentifierDatabricks table identifier
oneof id.airflow_dag AirflowDagIdentifierAirflow DAG identifier
oneof id.airflow_task AirflowTaskIdentifierAirflow task identifier within a given DAG
oneof id.custom CustomIdentifierCustom identifier to be used with all custom created entities
oneof id.dataproduct DataproductIdentifierDataproduct identifier
oneof id.synq_path SynqPathIdentifierSynqPath identifier

MysqlTableIdentifier

(synq.entities.v1.MysqlTableIdentifier)

FieldTypeDescription
host stringMysql host inclusive of port
schema stringMysql database
table stringMysql table

PostgresTableIdentifier

(synq.entities.v1.PostgresTableIdentifier)

FieldTypeDescription
host stringPostgres host inclusive of port
database stringPostgres database
schema stringPostgres schema
table stringPostgres table

RedshiftTableIdentifier

(synq.entities.v1.RedshiftTableIdentifier)

FieldTypeDescription
cluster stringRedshift cluster
database stringRedshift database
schema stringRedshift schema
table stringRedshift table

SnowflakeTableIdentifier

(synq.entities.v1.SnowflakeTableIdentifier)

FieldTypeDescription
account stringSnowflake account
database stringSnowflake database
schema stringSnowflake schema
table stringSnowflake table

SynqPathIdentifier

(synq.entities.v1.SynqPathIdentifier)

FieldTypeDescription
path stringSynq path that identifies the Synq entity, needs to be one of supported paths

Entity

(synq.entities.v1.Entity)

Entity is the base type for all entities in the system. Its always attached to a given client workspace and can be uniquely identified by its id.

FieldTypeDescription
workspace stringImmutable workspace identifier that this entity belongs to.
id IdentifierUnique identifier for this entity.
name stringName of the entity that is displayed to the user.
description stringDescription of the entity that is displayed in relevant contexts.
created_at TimestampTimestamp when the entity was created.
updated_at TimestampTimestamp when the entity was last updated.
deleted_at TimestampTimestamp when the entity was deleted. If this is set, the entity is considered deleted.
synq_path stringReturn only value. Synq Path to the entity. This is a unique path that can be used to access the entity in the Synq’s UI.
synq_catalog_url stringReturn only value. Synq URL to the entity. This is a unique URL that can be used to access the entity in the Synq’s UI.

Enums

Scope

(synq.v1.Scope)

NameNumberDescription
SCOPE_UNSPECIFIED0none
SCOPE_ENTITY_READ1none
SCOPE_ENTITY_EDIT2none
SCOPE_LINEAGE_READ10none
SCOPE_LINEAGE_EDIT11none
SCOPE_STATUS_READ20none

IssueStatus

(synq.status.v1.IssueStatus)

NameNumberDescription
ISSUE_STATUS_UNSPECIFIED0none
ISSUE_STATUS_OK10none
ISSUE_STATUS_WARN20none
ISSUE_STATUS_ERROR30none
ISSUE_STATUS_CRITICAL40none

CllState

(synq.lineage.v1.CllState)

NameNumberDescription
CLL_STATE_UNSPECIFIED0Unspecified state.
CLL_STATE_PARSE_FAILED1Parsing of the asset SQL failed. No upstream dependencies can be found.
CLL_STATE_EXTRACTION_FAILED2Extraction of the asset SQL failed. Some unsupported SQL features may be used. Some details might be missing.
CLL_STATE_RESOLUTION_FAILED3Not all columns or tables were found upstream, lineage is not complete.
CLL_STATE_OK10No known issues present.

NodePosition

(synq.lineage.v1.NodePosition)

NameNumberDescription
NODE_POSITION_UNSPECIFIED0none
NODE_POSITION_START_NODE1Node is one of the requested start point.
NODE_POSITION_UPSTREAM2Node is upstream of the requested start point.
NODE_POSITION_DOWNSTREAM3Node is downstream of the requested start point.

LineageDirection

(synq.lineage.v1.LineageDirection)

Direction of the lineage to query.

NameNumberDescription
LINEAGE_DIRECTION_UNSPECIFIED0none
LINEAGE_DIRECTION_UPSTREAM1none
LINEAGE_DIRECTION_DOWNSTREAM2none
LINEAGE_DIRECTION_UPSTREAM_DOWNSTREAM3none