make package compatible with Azure Synapse and Microsoft Fabric#529
make package compatible with Azure Synapse and Microsoft Fabric#529sdebruyn wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR extends dbt package compatibility to support Azure Synapse and Microsoft Fabric, which inherit from the SQL Server adapter. The changes replace SQL Server-specific conditional logic to include these new target types.
- Updates conditional logic throughout the codebase to include "synapse" and "fabric" alongside "sqlserver"
- Replaces SQL Server-specific macro implementations with Fabric equivalents
- Adds type mappings for the new database targets in configuration files
Reviewed Changes
Copilot reviewed 38 out of 38 changed files in this pull request and generated 12 comments.
Show a summary per file
| File | Description |
|---|---|
| models/staging/*.sql | Updated target type conditionals to include synapse and fabric |
| models/sources/*.sql | Updated target type conditionals to include synapse and fabric |
| models/fct_*.sql | Updated target type conditionals to include synapse and fabric |
| models/dim_*.sql | Updated target type conditionals to include synapse and fabric |
| macros/upload_results/*.sql | Renamed sqlserver__ macros to fabric__ and updated conditionals |
| macros/upload_individual_datasets/*.sql | Renamed sqlserver__ macros to fabric__ |
| macros/integration_tests/safe_cast.sql | Added type mappings for synapse and fabric |
| integration_test_project/seeds/seeds.yml | Updated column type conditional |
| dbt_project.yml | Updated persist_docs configuration conditionals |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
The copilot reviews are useless. The SQL Server adapter inherits from Fabric adapter. |
|
Hi @sdebruyn , We have tried your Fabric Support branch in a project and noticed that the It seems that the corresponding adapter variables are empty? These two values are set here and leverage the Is this an adapter issue or something in your Fabric support? Best regards |
Fixes brooklyn-data#521 — the default type_json/type_array use api.Column.translate_type("string") which on SQL Server/Fabric resolves to varchar(8000), causing truncation errors when uploading large JSON payloads. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Hi @tkirschke, thanks for testing this! The The timing data comes from dbt-core's task runner ( Could you check:
Also, if you're using Microsoft's |
Documents the package, dispatch config, required +file_format: delta setting, dbt-utils dependency, and the on-run-end upload hook. Notes Lakehouse compatibility (integration-tested) and links to upstream PR brooklyn-data/dbt_artifacts#529 for Synapse/Fabric DW support. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Documents the package, dispatch config, required +file_format: delta setting, dbt-utils dependency, and the on-run-end upload hook. Notes Lakehouse compatibility (integration-tested) and links to upstream PR brooklyn-data/dbt_artifacts#529 for Synapse/Fabric DW support. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Overview
This adds compatibility for Azure Synapse and Microsoft Fabric.
The new version of the sqlserver adapter inherits from the fabric adapter. So in the macros, we can replace
sqlserver__withfabric__.Changes
sqlserver__dispatch macros tofabric__(sqlserver inherits from fabric, so dispatch finds these via the adapter type hierarchy)fabric__type_json()andfabric__type_array()returningvarchar(max)— fixes [Bug]: Database Error ('22001', '[22001] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server] String or binary data would be truncated. #521 (truncation errors fromvarchar(8000))target.typeconditionals to include"synapse"and"fabric"alongside"sqlserver"Update type - breaking / non-breaking
What does this solve?
varchar(8000)instead ofvarchar(max), causing truncation when uploading large payloadsWhat databases have you tested with?