> ## Documentation Index > Fetch the complete documentation index at: https://docs.synq.io/llms.txt > Use this file to discover all available pages before exploring further. # IngestQueryLogs ## OpenAPI ````yaml /api-reference/openapi.yaml post /api/datawarehouse/v1/connection/{connection_id}/upload/{upload_id}/query-logs openapi: 3.1.0 info: version: '1.0' title: SYNQ servers: - url: https://developer.synq.io - url: https://api.us.synq.io security: - bearerAuth: [] tags: - name: synq.issues.issues.v1.IssuesService description: IssuesService is a service for managing Issues. - name: synq.issues.v2.IssuesService description: IssuesService is a service for managing Issues. - name: synq.incidents.v1.IncidentsService description: IncidentsService is a service for managing Incidents. - name: synq.alerts.services.v1.AlertsService description: AlertsService provides operations for managing alert configurations. - name: synq.agent.recon.v1.YamlService description: >- YamlService provides stateless conversion between YAML config format and proto. This is useful for UI editors that need to display/edit suite configs as YAML. - name: synq.agent.recon.v1.SuiteDeploymentService description: |- SuiteDeploymentService is the platform plane of reconciliation. SuiteConfigService is the developer/authoring sandbox (edit, version, run ad-hoc with your own credentials — preview-only, no Entity, no Run, no Issue). A deployment is what SYNQ actually runs on the workspace's behalf: a frozen snapshot of a suite config, mapped to workspace-level SYNQ integrations, optionally scheduled or triggerable by API. Identity: every deployment has a stable deployment_id (UUIDv7) assigned on first promote. It survives re-promotes, schedule/mapping changes, and pause/resume. Downstream AssetCommand / RunCommand publication keys off deployment_id so Entity identity does not fragment on promote churn. - name: synq.agent.sre.v1.TriageService description: >- Service for managing issue triage operations, allowing LLM agents to conclude investigations and record evidence during the triage process. - name: synq.agent.sre.v1.LlmService description: Service for evaluating LLM requests and producing structured output. - name: synq.agent.sre.v1.FeatureRequestService description: >- FeatureRequestService allows MCP clients to submit feature requests when users encounter missing capabilities. This is a last-resort service — it should only be used when no existing tool can fulfill the user's request. - name: synq.integrations.v1.IntegrationsService description: |- IntegrationsService manages connections from Coalesce Quality to your data systems (warehouses, databases, and transformation tools). Concurrency: every integration carries an opaque `etag`. Read it from `GetIntegration` / `ListIntegrations`, then pass it back on `UpdateIntegration` / `DeleteIntegration` to ensure you modify the version you last saw. A stale etag is rejected with ABORTED (HTTP 409). Omit the etag for last-write-wins. Quota: each workspace has a limit on the number of integrations. Creating beyond the limit is rejected with RESOURCE_EXHAUSTED. Secrets: credential fields (passwords, tokens, keys) are write-only. They are masked (returned empty) on every read. On update, omit a secret to keep it, send a new value to rotate it, or send an explicit empty string to clear it (where the field is `optional`). - name: synq.auth.iam.v1.IamService - name: synq.monitors.predictions.v1.MonitorPredictionsService description: Access to anomaly detection model predictions and raw metric timeseries. - name: synq.monitors.info.v1.MonitorInfoService - name: synq.monitors.custom_monitors.v1.CustomMonitorsService - name: synq.monitors.history.v1.HistoryService - name: synq.monitors.automated_monitors.v1.DeploymentRulesService - name: synq.entities.executions.v2.EntityExecutionsService description: >- EntityExecutionsService provides read-only access to entity execution history. This service allows customers to retrieve information about all executions that happened on their entities, including execution status, timing, and messages. Use cases: - Retrieve execution history for specific entities - Filter executions by time range, status, or execution type - Get aggregated summaries of execution activity - Track execution trends and patterns - name: synq.datachecks.sqltests.v1.SqlTestsService description: SqlTestsService is a service for managing SqlTests. - name: synq.datachecks.v1.TriggerService description: TriggerService provides synchronous execution of datachecks on entities. - name: synq.datachecks.testsuggestions.v1.TestSuggestionsService - name: synq.extensions.atlan.provider.v1.AtlanProviderService - name: synq.extensions.atlan.integrations.v1.AtlanIntegrationService - name: synq.extensions.atlan.workflows.v1.AtlanWorkflowService - name: synq.savedviews.v1.SavedViewsService description: >- SavedViewsService manages saved views — named, reusable selections over your entities (and other surfaces) with display configuration, sharing and per-user pinning. A saved view is owned by the user who created it. It can be kept private, shared with the whole workspace, or granted to specific users. Reads return, for the calling user, how each view relates to them (mine / shared / granted) and what they are allowed to do with it. Workspace administrators can manage any view in the workspace. - name: synq.platforms.v1.PlatformsService description: PlatformsService is a service for managing Platforms and Integrations. - name: synq.queries.v1.NLQueryService description: >- NLQueryService generates structured Query protos from natural language descriptions using an LLM. - name: synq.schedule.v1.ScheduleService description: ScheduleService provides schedule evaluation utilities. - name: synq.entities.executions.v1.EntityExecutionsService description: 'Deprecated: Use [synq.entities.custom.v1.EntityExecutionsService] instead' - name: synq.entities.status.v1.EntityIncidentsService description: EntityIncidentsService is the service which retrieves entity status. - name: synq.entities.status.v1.EntityIssuesService description: EntityIssuesService is the service which retrieves entity issues status. - name: synq.entities.orchestration.v1.OrchestrationService description: >- OrchestrationService provides information about orchestration relationships between entities. This includes relationships between Airflow tasks and transformation models (dbt, SQLMesh), as well as task-to-task dependencies. - name: synq.entities.coordinates.v1.DatabaseCoordinatesService description: >- DatabaseCoordinatesService is a service for getting database coordinates of Entities. - name: synq.entities.checks.v1.ChecksCategoriesService description: |- ChecksCategoriesService lets workspace admins set explicit category overrides on individual checks. An explicit category is the authoritative category for a check — it takes precedence over the categories computed by the workspace's categorisation rules. It is a public API so customers can manage check categories programmatically; the same service is also mounted on the internal API. The workspace and the acting identity are always taken from the request context, never from the payload. - name: synq.entities.sql_insights.v1.SqlInsightsService description: >- SqlInsightsService exposes analytical information about the SQL used across a workspace's entities. It answers questions such as "which SQL constructs are used in my warehouse, and how often" and "what SQL constructs does this specific entity use", without requiring the caller to re-parse any SQL. - name: synq.entities.custom.v1.RelationshipsService description: >- RelationshipsService allow management of relationships between entities. Relationships can be created, updated, and deleted between 2 custom entities, or between a custom entity and Coalesce Quality native entity.enum There is no option to create relationships between 2 Coalesce Quality native entities (dbt model, BI dashboard, etc.). - name: synq.entities.custom.v1.TypesService description: TypesService is a service for managing custom entity types. - name: synq.entities.custom.v1.GroupsService description: >- It eliminates the need to keep state on client side to remember which assets were already created and which should be deleted. The server will keep track of the current state of the group and client can always send the intended new state. The server will calculate the diff and entities that are no longer present in the group will be removed. Example: 1. group has entities A, B, C at time t1 2. client sends group with entities B, C, D at time t2 3. server will remove entity A from the system and update the current state of the group to B, C, D The service is designed to be idempotent and can be called multiple times with the same state without causing any side effects. - name: synq.entities.custom.v1.EntityExecutionsService - name: synq.entities.custom.v1.EntitiesService description: >- custom.EntitiesService is a service for managing custom entities. Entities can represent various data platform concepts such as services, consumers, applications or data pipelines that are not natively available in Coalesce Quality. Entities are identified by a unique identifier and can be created, updated, read and deleted. - name: synq.entities.custom.v1.FeaturesService - name: synq.entities.impact.v1.ImpactService - name: synq.entities.schemas.v1.SchemaMismatchesService description: >- SchemaMismatchesService provides access to schema drift information between data platform tables and their definitions (e.g., dbt models). - name: synq.entities.schemas.v1.SchemasService description: EntitiesService is a service for retriving any entity. - name: synq.entities.constraints.v1.TableConstraintsService description: >- TableConstraintsService provides access to table constraint and index information. - name: synq.entities.changes.v1.ChangesService description: >- ChangesService provides functionality to track and retrieve all types of changes to data entities. This unified service returns ALL change types for an entity: - Git commits: Changes to code files (dbt models, SQL files) tracked in version control - Schema changes: Database schema modifications (columns added/removed/changed) - SQL definition changes: View/materialized view definition updates detected by Coalesce Quality Changes are returned with structured metadata including: - For git commits: structured statistics (directories, file types, top changes) - For schema changes: detailed column-level diffs - For SQL changes: before/after SQL definitions Use cases: - "What changed in the last week for table X?" → Returns git commits, schema changes, SQL changes - "Show me all commits affecting this dbt model" → Returns git commits with lineage context - "What schema changes happened to this table?" → Returns schema changes detected by Coalesce Quality - name: synq.entities.lineage.v1.LineageService description: |- LineageService allows you to fetch: * Entity level lineage from a starting point of one or more entities. * Column Level lineage from a starting point of multiple columns of a single entity. - name: synq.entities.entities.v1.EntitiesService description: EntitiesService is a service for retriving any entity. - name: synq.entities.annotations.v1.AnnotationsService description: >- AnnotationsService provides operations for managing and querying entity annotations. Annotations are key-value pairs that can be attached to entities for categorization and filtering. - name: synq.entities.code.v1.CodeService description: >- CodeService is a service for retrieving code associated with entities in the system. It provides functionality to access and manage code artifacts such as SQL queries, Python scripts, dbt models, and other code configurations that are part of Coalesce Quality entities. - name: synq.entities.resolve.v1.IdentifierResolveService description: >- IdentifierResolveService resolves identifiers to their Coalesce Quality paths and identities. - name: synq.ingest.airflow.v1.AirflowLogsService - name: synq.ingest.cloudwatch.v1.CloudwatchService - name: synq.ingest.openlineage.v1.OpenlineageService - name: synq.ingest.dwh.v1.DwhService - name: synq.dataproducts.v1.DataproductsService description: DataproductsService can be used to manage data products. - name: synq.git.commits.v1.CommitsService paths: /api/datawarehouse/v1/connection/{connection_id}/upload/{upload_id}/query-logs: post: tags: - synq.ingest.dwh.v1.DwhService summary: IngestQueryLogs operationId: synq.ingest.dwh.v1.DwhService.IngestQueryLogs parameters: - name: connection_id in: path required: true schema: type: string title: connection_id - name: upload_id in: path required: true schema: type: string title: upload_id format: uuid requestBody: content: application/json: schema: type: object properties: queryLogs: type: array items: $ref: '#/components/schemas/synq.ingest.dwh.v1.QueryLog' title: query_logs title: IngestQueryLogsRequest additionalProperties: false required: true responses: '200': description: Success content: application/json: schema: $ref: >- #/components/schemas/synq.ingest.dwh.v1.IngestQueryLogsResponse components: schemas: synq.ingest.dwh.v1.QueryLog: type: object properties: workspace: type: string description: Workspace and integration identifiers (for multi-tenancy) integrationId: type: string connectionId: type: string description: Empty for direct connections, populated for agent uploads nullable: true queryId: type: string description: Query identifiers createdAt: $ref: '#/components/schemas/google.protobuf.Timestamp' startedAt: $ref: '#/components/schemas/google.protobuf.Timestamp' description: Query start time (optional, uses created_at if not set) nullable: true finishedAt: $ref: '#/components/schemas/google.protobuf.Timestamp' description: Query finish time (optional, uses created_at if not set) nullable: true sql: type: string description: >- Query content SQL text (may be obfuscated based on sql_obfuscation_mode) sqlHash: type: string description: >- SHA256 hash of original SQL for deduplication (computed during storage if not provided) nullable: true normalizedQueryHash: type: string description: >- Hash of normalized query for lineage caching (empty if not available from platform) nullable: true sqlDialect: type: string description: SQL dialect (e.g., "snowflake", "bigquery", "clickhouse") queryType: type: string description: >- Platform-specific query type (e.g., "CREATE_TABLE_AS_SELECT", "SELECT") status: type: string description: 'Execution status: "SUCCESS", "FAILED", "CANCELED"' dwhContext: $ref: '#/components/schemas/synq.ingest.dwh.v1.QueryLogDwhContext' description: DWH execution context nullable: true sqlObfuscationMode: $ref: '#/components/schemas/synq.ingest.dwh.v1.SqlObfuscationMode' description: Obfuscation and parsing hints hasCompleteNativeLineage: type: boolean description: If true, native lineage is complete and SQL parsing can be skipped isTruncated: type: boolean description: If true, SQL was truncated by the warehouse metadata: $ref: '#/components/schemas/google.protobuf.Struct' description: |- Platform-specific metadata (arbitrary key-value pairs) Contains execution metrics, costs, etc. depending on the platform nullable: true nativeLineage: $ref: '#/components/schemas/synq.ingest.dwh.v1.QueryLogNativeLineage' description: Native lineage from the platform (if available) nullable: true title: QueryLog required: - queryId - createdAt additionalProperties: false description: >- QueryLog represents a single query execution log entry from a data warehouse. This is the format used for ingesting query logs via the DWH agent. The structure mirrors the internal QueryLogRawRow with protobuf serialization. synq.ingest.dwh.v1.IngestQueryLogsResponse: type: object title: IngestQueryLogsResponse additionalProperties: false google.protobuf.Timestamp: type: string examples: - '2023-01-15T01:30:15.01Z' - '2024-12-25T12:00:00Z' format: date-time description: >- A Timestamp represents a point in time independent of any time zone or local calendar, encoded as a count of seconds and fractions of seconds at nanosecond resolution. The count is relative to an epoch at UTC midnight on January 1, 1970, in the proleptic Gregorian calendar which extends the Gregorian calendar backwards to year one. All minutes are 60 seconds long. Leap seconds are "smeared" so that no leap second table is needed for interpretation, using a [24-hour linear smear](https://developers.google.com/time/smear). The range is from 0001-01-01T00:00:00Z to 9999-12-31T23:59:59.999999999Z. By restricting to that range, we ensure that we can convert to and from [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) date strings. # Examples Example 1: Compute Timestamp from POSIX `time()`. Timestamp timestamp; timestamp.set_seconds(time(NULL)); timestamp.set_nanos(0); Example 2: Compute Timestamp from POSIX `gettimeofday()`. struct timeval tv; gettimeofday(&tv, NULL); Timestamp timestamp; timestamp.set_seconds(tv.tv_sec); timestamp.set_nanos(tv.tv_usec * 1000); Example 3: Compute Timestamp from Win32 `GetSystemTimeAsFileTime()`. FILETIME ft; GetSystemTimeAsFileTime(&ft); UINT64 ticks = (((UINT64)ft.dwHighDateTime) << 32) | ft.dwLowDateTime; // A Windows tick is 100 nanoseconds. Windows epoch 1601-01-01T00:00:00Z // is 11644473600 seconds before Unix epoch 1970-01-01T00:00:00Z. Timestamp timestamp; timestamp.set_seconds((INT64) ((ticks / 10000000) - 11644473600LL)); timestamp.set_nanos((INT32) ((ticks % 10000000) * 100)); Example 4: Compute Timestamp from Java `System.currentTimeMillis()`. long millis = System.currentTimeMillis(); Timestamp timestamp = Timestamp.newBuilder().setSeconds(millis / 1000) .setNanos((int) ((millis % 1000) * 1000000)).build(); Example 5: Compute Timestamp from Java `Instant.now()`. Instant now = Instant.now(); Timestamp timestamp = Timestamp.newBuilder().setSeconds(now.getEpochSecond()) .setNanos(now.getNano()).build(); Example 6: Compute Timestamp from current time in Python. timestamp = Timestamp() timestamp.GetCurrentTime() # JSON Mapping In JSON format, the Timestamp type is encoded as a string in the [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt) format. That is, the format is "{year}-{month}-{day}T{hour}:{min}:{sec}[.{frac_sec}]Z" where {year} is always expressed using four digits while {month}, {day}, {hour}, {min}, and {sec} are zero-padded to two digits each. The fractional seconds, which can go up to 9 digits (i.e. up to 1 nanosecond resolution), are optional. The "Z" suffix indicates the timezone ("UTC"); the timezone is required. A proto3 JSON serializer should always use UTC (as indicated by "Z") when printing the Timestamp type and a proto3 JSON parser should be able to accept both UTC and other timezones (as indicated by an offset). For example, "2017-01-15T01:30:15.01Z" encodes 15.01 seconds past 01:30 UTC on January 15, 2017. In JavaScript, one can convert a Date object to this format using the standard [toISOString()](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/toISOString) method. In Python, a standard `datetime.datetime` object can be converted to this format using [`strftime`](https://docs.python.org/2/library/time.html#time.strftime) with the time format spec '%Y-%m-%dT%H:%M:%S.%fZ'. Likewise, in Java, one can use the Joda Time's [`ISODateTimeFormat.dateTime()`]( http://www.joda.org/joda-time/apidocs/org/joda/time/format/ISODateTimeFormat.html#dateTime%2D%2D ) to obtain a formatter capable of generating timestamps in this format. synq.ingest.dwh.v1.QueryLogDwhContext: type: object properties: instance: type: string description: Instance identifier (account, workspace_url, host, etc.) nullable: true database: type: string description: Database/catalog name nullable: true schema: type: string description: Schema name nullable: true warehouse: type: string description: Warehouse identifier (Snowflake, Databricks) nullable: true user: type: string description: User who executed the query nullable: true role: type: string description: Role used for execution nullable: true cluster: type: string description: Cluster identifier (Redshift, ClickHouse) nullable: true title: QueryLogDwhContext additionalProperties: false description: |- QueryLogDwhContext represents the execution context of a query. Contains information about where and by whom the query was executed. Platform-specific mappings (Instance, Database, Schema): - Snowflake: account, database_name, schema_name - Databricks: workspace_url, catalog_name, schema_name - BigQuery: "", project_id, dataset_id - Redshift: host, database_name, schema_name - Postgres: host, database_name, schema_name - Trino: host, catalog, schema - MySQL: "", host, schema_name - ClickHouse: hostname, database_name, "" (2-level: hostname.database) - DuckDB: motherduck_account, database_name, schema_name synq.ingest.dwh.v1.SqlObfuscationMode: type: string title: SqlObfuscationMode enum: - SQL_OBFUSCATION_MODE_NONE - SQL_OBFUSCATION_MODE_REDACT_LITERALS description: >- SqlObfuscationMode represents the level of SQL obfuscation applied to query logs. This is critical for on-premise deployments where customers want to prevent sensitive data in SQL queries from being sent to Coalesce Quality backend. google.protobuf.Struct: type: object additionalProperties: $ref: '#/components/schemas/google.protobuf.Value' description: |- `Struct` represents a structured data value, consisting of fields which map to dynamically typed values. In some languages, `Struct` might be supported by a native representation. For example, in scripting languages like JS a struct is represented as an object. The details of that representation are described together with the proto support for the language. The JSON representation for `Struct` is JSON object. synq.ingest.dwh.v1.QueryLogNativeLineage: type: object properties: inputTables: type: array items: $ref: '#/components/schemas/synq.ingest.dwh.v1.Fqn' description: Tables read by the query outputTables: type: array items: $ref: '#/components/schemas/synq.ingest.dwh.v1.Fqn' description: Tables written to by the query title: QueryLogNativeLineage additionalProperties: false description: >- QueryLogNativeLineage contains lineage information provided natively by the data warehouse. Not all platforms provide this - when available, it can be more accurate than SQL parsing. google.protobuf.Value: oneOf: - type: 'null' - type: number - type: string - type: boolean - type: array - type: object additionalProperties: true description: |- `Value` represents a dynamically typed value which can be either null, a number, a string, a boolean, a recursive struct value, or a list of values. A producer of value is expected to set one of these variants. Absence of any variant indicates an error. The JSON representation for `Value` is JSON value. synq.ingest.dwh.v1.Fqn: type: object properties: instanceName: type: string databaseName: type: string schemaName: type: string objectName: type: string title: Fqn required: - objectName additionalProperties: false securitySchemes: bearerAuth: type: http scheme: bearer ````