Skip to content

make package compatible with Azure Synapse and Microsoft Fabric#529

Open
sdebruyn wants to merge 2 commits into
brooklyn-data:mainfrom
sdebruyn:fabric-fixes
Open

make package compatible with Azure Synapse and Microsoft Fabric#529
sdebruyn wants to merge 2 commits into
brooklyn-data:mainfrom
sdebruyn:fabric-fixes

Conversation

@sdebruyn
Copy link
Copy Markdown
Contributor

@sdebruyn sdebruyn commented Sep 23, 2025

Overview

This adds compatibility for Azure Synapse and Microsoft Fabric.

The new version of the sqlserver adapter inherits from the fabric adapter. So in the macros, we can replace sqlserver__ with fabric__.

Changes

Update type - breaking / non-breaking

  • New features (non-breaking change)

What does this solve?

What databases have you tested with?

  • Microsoft Fabric (Data Warehouse)

Copilot AI review requested due to automatic review settings September 23, 2025 15:22
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
@sdebruyn sdebruyn had a problem deploying to Approve Integration Tests September 23, 2025 15:22 — with GitHub Actions Failure
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR extends dbt package compatibility to support Azure Synapse and Microsoft Fabric, which inherit from the SQL Server adapter. The changes replace SQL Server-specific conditional logic to include these new target types.

  • Updates conditional logic throughout the codebase to include "synapse" and "fabric" alongside "sqlserver"
  • Replaces SQL Server-specific macro implementations with Fabric equivalents
  • Adds type mappings for the new database targets in configuration files

Reviewed Changes

Copilot reviewed 38 out of 38 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
models/staging/*.sql Updated target type conditionals to include synapse and fabric
models/sources/*.sql Updated target type conditionals to include synapse and fabric
models/fct_*.sql Updated target type conditionals to include synapse and fabric
models/dim_*.sql Updated target type conditionals to include synapse and fabric
macros/upload_results/*.sql Renamed sqlserver__ macros to fabric__ and updated conditionals
macros/upload_individual_datasets/*.sql Renamed sqlserver__ macros to fabric__
macros/integration_tests/safe_cast.sql Added type mappings for synapse and fabric
integration_test_project/seeds/seeds.yml Updated column type conditional
dbt_project.yml Updated persist_docs configuration conditionals

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread macros/upload_results/insert_into_metadata_table.sql
Comment thread macros/upload_individual_datasets/upload_tests.sql
Comment thread macros/upload_individual_datasets/upload_test_executions.sql
Comment thread macros/upload_individual_datasets/upload_sources.sql
Comment thread macros/upload_individual_datasets/upload_snapshots.sql
Comment thread macros/upload_individual_datasets/upload_seed_executions.sql
Comment thread macros/upload_individual_datasets/upload_models.sql
Comment thread macros/upload_individual_datasets/upload_model_executions.sql
Comment thread macros/upload_individual_datasets/upload_invocations.sql Outdated
Comment thread macros/upload_individual_datasets/upload_exposures.sql
@sdebruyn
Copy link
Copy Markdown
Contributor Author

The copilot reviews are useless. The SQL Server adapter inherits from Fabric adapter.
@mtcarlone Could you take a look please? :)

@tkirschke
Copy link
Copy Markdown

Hi @sdebruyn ,

We have tried your Fabric Support branch in a project and noticed that the model_executions lack information regarding compile_started_at and query_completed_at - the corresponding columns are null.

It seems that the corresponding adapter variables are empty? These two values are set here and leverage the model.timing attribute.

Is this an adapter issue or something in your Fabric support?

Best regards
Tim

sdebruyn and others added 2 commits May 17, 2026 14:06
Fixes brooklyn-data#521 — the default type_json/type_array use api.Column.translate_type("string")
which on SQL Server/Fabric resolves to varchar(8000), causing truncation errors
when uploading large JSON payloads.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sdebruyn
Copy link
Copy Markdown
Contributor Author

sdebruyn commented May 17, 2026

Hi @tkirschke, thanks for testing this!

The compile_started_at and query_completed_at columns being null is not caused by this PR or the Fabric adapter. I've verified that the dbt-fabric adapter correctly produces compile and execute timing entries in model.timing — confirmed by checking NodeFinished events in test logs, which include both compile.started_at and execute.completed_at.

The timing data comes from dbt-core's task runner (BaseRunner.compile_and_execute()), not from the adapter, so it should work identically across all adapters.

Could you check:

  1. Which version of dbt-core are you using?
  2. Does your run_results.json contain timing entries for the affected models?
  3. Are the affected models Python models or SQL models?

Also, if you're using Microsoft's dbt-fabric adapter, you might want to try my maintained fork instead — it has additional features and fixes: dbt-fabric-samdebruyn on PyPI / GitHub repo. The timing issue might be specific to the upstream adapter version you're using.

sdebruyn added a commit to sdebruyn/dbt-fabric that referenced this pull request May 18, 2026
Documents the package, dispatch config, required +file_format: delta
setting, dbt-utils dependency, and the on-run-end upload hook. Notes
Lakehouse compatibility (integration-tested) and links to upstream PR
brooklyn-data/dbt_artifacts#529 for Synapse/Fabric DW support.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sdebruyn added a commit to sdebruyn/dbt-fabric that referenced this pull request May 18, 2026
Documents the package, dispatch config, required +file_format: delta
setting, dbt-utils dependency, and the on-run-end upload hook. Notes
Lakehouse compatibility (integration-tested) and links to upstream PR
brooklyn-data/dbt_artifacts#529 for Synapse/Fabric DW support.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Database Error ('22001', '[22001] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server] String or binary data would be truncated.

3 participants