From f0904613b1d2fee80f35ec0a0d34c8e98b9a10e7 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 6 May 2026 11:12:49 -0700 Subject: [PATCH 1/8] Add FOCUS schema tests and changelog entries (#2120) - HubsFocusSchemas.Tests.ps1: 108 Pester tests asserting the FOCUS 1.3 and 1.4-preview schema additions land where expected: Costs_raw additions, ContractCommitment_raw definition with both 1.3 and 1.4 columns, presence of versioned transforms and final tables in IngestionSetup_v1_3 / v1_4, presence of versioned hub functions in HubSetup_v1_3 / v1_4, the union arms in Costs_v1_3 and Costs_v1_4, the Latest aliases pinned to v1_3 (not preview v1_4), and the Bicep / build-config wiring. - changelog.md: adds v15.0.0 hub entries describing the FOCUS 1.3 GA and 1.4-preview schema additions, the new ContractCommitment dataset, the unversioned alias retarget, and the upgrade-procedure refresh. Adds a plugin entry for #2119. Updates ms.date to today. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs-mslearn/toolkit/changelog.md | 14 +- .../Tests/Unit/HubsFocusSchemas.Tests.ps1 | 228 ++++++++++++++++++ 2 files changed, 241 insertions(+), 1 deletion(-) create mode 100644 src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index dc22e673f..651aeb6b5 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 04/29/2026 +ms.date: 05/06/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit @@ -31,6 +31,18 @@ The following section lists features and enhancements that are currently in deve - Added Claude Code plugin with skills for FinOps hubs and Azure Cost Management ([#2043](https://github.com/microsoft/finops-toolkit/pull/2043)). - Added 4 agents (CFO, FinOps practitioner, database query, hubs agent), 5 commands (`/ftk-hubs-connect`, `/ftk-hubs-healthCheck`, `/ftk-mom-report`, `/ftk-ytd-report`, `/ftk-cost-optimization`), and an output style. - Linked to the existing KQL query catalog in `src/queries/` from the plugin. +- **Changed** + - Updated plugin skill files to reflect FOCUS 1.3 / 1.4-preview hub schemas, the new `ContractCommitment()` function, and the deprecation of `ProviderName` / `PublisherName` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + +### FinOps hubs v15.0.0 + +- **Added** + - Added FOCUS 1.3 hub schema (`v1_3`). New cost and usage columns: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`. New supplemental dataset: `ContractCommitment` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + - Added FOCUS 1.4-preview hub schema (`v1_4`). Drops `ProviderName` and `PublisherName` from cost and usage. Expands `ContractCommitment` by 14 columns (`BenefitCategory`, `ContractCommitmentApplicability`, `Created`, `DiscountPercentage`, `DurationType`, `FulfillmentInterval`, `LastUpdated`, `LifecycleStatus`, `Model`, `OfferCategory`, `PaymentInterval`, `PaymentModel`, `PaymentUpfrontPercentage`, `PricingCurrencyContractCommitmentCost`). Marked **preview** because FOCUS 1.4 is still in working draft and may change before ratification. + - Added unversioned `ContractCommitment()` function aliasing to `ContractCommitment_v1_3()`. +- **Changed** + - Retargeted unversioned `Costs()`, `Prices()`, `CommitmentDiscountUsage()`, `Recommendations()`, and `Transactions()` aliases to their `_v1_3` counterparts. Unversioned aliases stay pinned to the latest GA schema (v1_3) — `_v1_4` is opt-in only while in preview. + - Refreshed the canonical "add a new FOCUS version" procedure in [src/templates/finops-hub/docs/README.md](https://github.com/microsoft/finops-toolkit/tree/dev/src/templates/finops-hub/docs/README.md) with multi-version-cycle, plugin, and changelog steps. ### Bicep Registry module pending updates diff --git a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 new file mode 100644 index 000000000..1e9e18432 --- /dev/null +++ b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 @@ -0,0 +1,228 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +Describe 'HubsFocusSchemas' { + + BeforeAll { + $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path + $scriptsPath = Join-Path $repoRoot 'src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts' + $rawTablesContent = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_RawTables.kql') -Raw + + $ingestionFiles = @{ + v1_0 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_0.kql') -Raw + v1_2 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_2.kql') -Raw + v1_3 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_3.kql') -Raw + v1_4 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_4.kql') -Raw + } + + $hubFiles = @{ + v1_0 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_0.kql') -Raw + v1_2 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_2.kql') -Raw + v1_3 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_3.kql') -Raw + v1_4 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_4.kql') -Raw + Latest = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_Latest.kql') -Raw + } + + $appBicep = Get-Content -Path (Join-Path $repoRoot 'src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep') -Raw + $buildConfig = Get-Content -Path (Join-Path $repoRoot 'src/templates/finops-hub/.build.config') -Raw + } + + Context 'FOCUS 1.3 columns in Costs_raw' { + + BeforeAll { + # Extract just the Costs_raw alter block (not Costs_final or any other table). + $script:costsRawBlock = if ($rawTablesContent -match '(?ms)\.alter table Costs_raw \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } + } + + It 'Costs_raw block was extracted' { + $costsRawBlock | Should -Not -BeNullOrEmpty + } + + It 'Adds <_> to Costs_raw' -ForEach @( + 'AllocatedMethodId', 'AllocatedMethodDetails', 'AllocatedResourceId', + 'AllocatedResourceName', 'AllocatedTags', 'ContractApplied', + 'ServiceProviderName', 'HostProviderName' + ) { + $costsRawBlock | Should -Match "(?m)^\s+$_\s*:" + } + } + + Context 'ContractCommitment_raw exists with FOCUS 1.3 + 1.4 columns' { + + BeforeAll { + # The Redefine-all-columns alter-table block is the second occurrence; match all and pick it. + $allBlocks = [regex]::Matches($rawTablesContent, '(?ms)\.alter table ContractCommitment_raw \(\r?\n(.*?)\r?\n\)') + $script:contractCommitmentRawBlock = if ($allBlocks.Count -ge 1) { $allBlocks[$allBlocks.Count - 1].Groups[1].Value } else { '' } + } + + It 'Defines ContractCommitment_raw' { + $rawTablesContent | Should -Match '\.alter table ContractCommitment_raw \(' + } + + It 'ContractCommitment_raw column block was extracted' { + $contractCommitmentRawBlock | Should -Not -BeNullOrEmpty + } + + It 'Includes FOCUS 1.3 column <_>' -ForEach @( + 'BillingCurrency', 'ContractCommitmentCategory', 'ContractCommitmentCost', + 'ContractCommitmentId', 'ContractCommitmentPeriodEnd', 'ContractCommitmentPeriodStart', + 'ContractCommitmentQuantity', 'ContractCommitmentType', 'ContractCommitmentUnit', + 'ContractId', 'ContractPeriodEnd', 'ContractPeriodStart', 'InvoiceIssuerName', + 'PricingCurrency' + ) { + $contractCommitmentRawBlock | Should -Match "(?m)^\s+$_\s*:" + } + + It 'Includes FOCUS 1.4 column <_>' -ForEach @( + 'BenefitCategory', 'ContractCommitmentApplicability', 'Created', + 'DiscountPercentage', 'DurationType', 'FulfillmentInterval', 'LastUpdated', + 'LifecycleStatus', 'Model', 'OfferCategory', 'PaymentInterval', 'PaymentModel', + 'PaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost' + ) { + $contractCommitmentRawBlock | Should -Match "(?m)^\s+$_\s*:" + } + } + + Context 'IngestionSetup_v1_3.kql' { + + BeforeAll { + $script:costsFinalV13Block = if ($ingestionFiles.v1_3 -match '(?ms)\.create-merge table Costs_final_v1_3 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } + } + + It 'Defines Costs_transform_v1_3' { + $ingestionFiles.v1_3 | Should -Match 'Costs_transform_v1_3\(\)' + } + + It 'Defines Costs_final_v1_3 table' { + $ingestionFiles.v1_3 | Should -Match '\.create-merge table Costs_final_v1_3' + } + + It 'Defines ContractCommitment_transform_v1_3' { + $ingestionFiles.v1_3 | Should -Match 'ContractCommitment_transform_v1_3\(\)' + } + + It 'Defines ContractCommitment_final_v1_3 table' { + $ingestionFiles.v1_3 | Should -Match '\.create-merge table ContractCommitment_final_v1_3' + } + + It 'Costs_final_v1_3 block was extracted' { + $costsFinalV13Block | Should -Not -BeNullOrEmpty + } + + It 'Costs_final_v1_3 includes FOCUS 1.3 column <_>' -ForEach @( + 'AllocatedMethodId', 'AllocatedMethodDetails', 'AllocatedResourceId', + 'AllocatedResourceName', 'AllocatedTags', 'ContractApplied', + 'ServiceProviderName', 'HostProviderName' + ) { + $costsFinalV13Block | Should -Match "(?m)^\s+$_\s*:" + } + + It 'Costs_final_v1_3 still includes deprecated <_> for back compat' -ForEach @( + 'ProviderName', 'PublisherName' + ) { + $costsFinalV13Block | Should -Match "(?m)^\s+$_\s*:" + } + } + + Context 'IngestionSetup_v1_4.kql' { + + BeforeAll { + $script:costsFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table Costs_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } + $script:contractCommitmentFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table ContractCommitment_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } + } + + It 'Defines Costs_transform_v1_4' { + $ingestionFiles.v1_4 | Should -Match 'Costs_transform_v1_4\(\)' + } + + It 'Defines Costs_final_v1_4 table' { + $ingestionFiles.v1_4 | Should -Match '\.create-merge table Costs_final_v1_4' + } + + It 'Costs_final_v1_4 block was extracted' { + $costsFinalV14Block | Should -Not -BeNullOrEmpty + } + + It 'Costs_final_v1_4 does NOT include deprecated <_> (removed in 1.4)' -ForEach @( + 'ProviderName', 'PublisherName' + ) { + $costsFinalV14Block | Should -Not -Match "(?m)^\s+$_\s*:" + } + + It 'ContractCommitment_final_v1_4 includes FOCUS 1.4 column <_>' -ForEach @( + 'BenefitCategory', 'ContractCommitmentApplicability', 'Created', + 'DiscountPercentage', 'DurationType', 'FulfillmentInterval', 'LastUpdated', + 'LifecycleStatus', 'Model', 'OfferCategory', 'PaymentInterval', 'PaymentModel', + 'PaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost' + ) { + $contractCommitmentFinalV14Block | Should -Match "(?m)^\s+$_\s*:" + } + } + + Context 'HubSetup_v1_3.kql' { + + It 'Defines <_>' -ForEach @( + 'CommitmentDiscountUsage_v1_3', 'ContractCommitment_v1_3', + 'Costs_v1_3', 'Prices_v1_3', 'Recommendations_v1_3', 'Transactions_v1_3' + ) { + $hubFiles.v1_3 | Should -Match "$_\(\)" + } + + It 'Costs_v1_3 unions all prior versions' -ForEach @( + 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_3' + ) { + $hubFiles.v1_3 | Should -Match "database\('Ingestion'\)\.$_" + } + } + + Context 'HubSetup_v1_4.kql (preview)' { + + It 'Defines <_>' -ForEach @( + 'CommitmentDiscountUsage_v1_4', 'ContractCommitment_v1_4', + 'Costs_v1_4', 'Prices_v1_4', 'Recommendations_v1_4', 'Transactions_v1_4' + ) { + $hubFiles.v1_4 | Should -Match "$_\(\)" + } + + It 'Costs_v1_4 unions all prior versions' -ForEach @( + 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_3', 'Costs_final_v1_4' + ) { + $hubFiles.v1_4 | Should -Match "database\('Ingestion'\)\.$_" + } + + It 'Marks itself as preview in docstring' { + $hubFiles.v1_4 | Should -Match 'FOCUS 1\.4-preview' + } + } + + Context 'HubSetup_Latest.kql aliases' { + + It 'Aliases <_> to v1_3 (latest GA, not v1_4 preview)' -ForEach @( + 'CommitmentDiscountUsage', 'Costs', 'Prices', 'Recommendations', 'Transactions', + 'ContractCommitment' + ) { + $hubFiles.Latest | Should -Match "(?ms)$_\(\)\s*\{\s*${_}_v1_3\(\)\s*\}" + } + + It 'Does NOT alias to v1_4 (preview must be opt-in)' { + $hubFiles.Latest | Should -Not -Match '_v1_4\(\)' + } + } + + Context 'Bicep wiring' { + + It 'app.bicep loads <_>' -ForEach @( + 'IngestionSetup_v1_3.kql', 'IngestionSetup_v1_4.kql', + 'HubSetup_v1_3.kql', 'HubSetup_v1_4.kql' + ) { + $appBicep | Should -Match ([regex]::Escape($_)) + } + + It '.build.config lists <_>' -ForEach @( + 'IngestionSetup_v1_3.kql', 'IngestionSetup_v1_4.kql', + 'HubSetup_v1_3.kql', 'HubSetup_v1_4.kql' + ) { + $buildConfig | Should -Match ([regex]::Escape($_)) + } + } +} From eba731fdecb9a7db4b8441dc1fbce7b18a122f20 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Sun, 17 May 2026 13:29:39 -0700 Subject: [PATCH 2/8] Add FOCUS 1.4 schema tests and changelog entries Updates Pester tests to validate v1_4 (not v1_3) schema files, removes tests for the eliminated intermediate v1_3 schema, and corrects HubSetup_Latest assertions to expect _v1_4() aliases. Updates changelog to reflect FOCUS 1.4 GA release without preview designation. Co-Authored-By: Claude --- docs-mslearn/toolkit/changelog.md | 9 +- .../Tests/Unit/HubsFocusSchemas.Tests.ps1 | 100 ++++-------------- 2 files changed, 25 insertions(+), 84 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 651aeb6b5..ceb1b0eba 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -32,16 +32,15 @@ The following section lists features and enhancements that are currently in deve - Added 4 agents (CFO, FinOps practitioner, database query, hubs agent), 5 commands (`/ftk-hubs-connect`, `/ftk-hubs-healthCheck`, `/ftk-mom-report`, `/ftk-ytd-report`, `/ftk-cost-optimization`), and an output style. - Linked to the existing KQL query catalog in `src/queries/` from the plugin. - **Changed** - - Updated plugin skill files to reflect FOCUS 1.3 / 1.4-preview hub schemas, the new `ContractCommitment()` function, and the deprecation of `ProviderName` / `PublisherName` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + - Updated plugin skill files to reflect FOCUS 1.4 hub schema, the new `ContractCommitment()` function, and the removal of `ProviderName` / `PublisherName` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). ### FinOps hubs v15.0.0 - **Added** - - Added FOCUS 1.3 hub schema (`v1_3`). New cost and usage columns: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`. New supplemental dataset: `ContractCommitment` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). - - Added FOCUS 1.4-preview hub schema (`v1_4`). Drops `ProviderName` and `PublisherName` from cost and usage. Expands `ContractCommitment` by 14 columns (`BenefitCategory`, `ContractCommitmentApplicability`, `Created`, `DiscountPercentage`, `DurationType`, `FulfillmentInterval`, `LastUpdated`, `LifecycleStatus`, `Model`, `OfferCategory`, `PaymentInterval`, `PaymentModel`, `PaymentUpfrontPercentage`, `PricingCurrencyContractCommitmentCost`). Marked **preview** because FOCUS 1.4 is still in working draft and may change before ratification. - - Added unversioned `ContractCommitment()` function aliasing to `ContractCommitment_v1_3()`. + - Added FOCUS 1.4 hub schema (`v1_4`). New cost and usage columns: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`. Removes deprecated `ProviderName` and `PublisherName`. New supplemental dataset: `ContractCommitment` with 28 columns ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + - Added unversioned `ContractCommitment()` function aliasing to `ContractCommitment_v1_4()`. - **Changed** - - Retargeted unversioned `Costs()`, `Prices()`, `CommitmentDiscountUsage()`, `Recommendations()`, and `Transactions()` aliases to their `_v1_3` counterparts. Unversioned aliases stay pinned to the latest GA schema (v1_3) — `_v1_4` is opt-in only while in preview. + - Retargeted unversioned `Costs()`, `Prices()`, `CommitmentDiscountUsage()`, `Recommendations()`, and `Transactions()` aliases to their `_v1_4` counterparts. - Refreshed the canonical "add a new FOCUS version" procedure in [src/templates/finops-hub/docs/README.md](https://github.com/microsoft/finops-toolkit/tree/dev/src/templates/finops-hub/docs/README.md) with multi-version-cycle, plugin, and changelog steps. ### Bicep Registry module pending updates diff --git a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 index 1e9e18432..f49ff5483 100644 --- a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 +++ b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 @@ -11,14 +11,12 @@ Describe 'HubsFocusSchemas' { $ingestionFiles = @{ v1_0 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_0.kql') -Raw v1_2 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_2.kql') -Raw - v1_3 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_3.kql') -Raw v1_4 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_4.kql') -Raw } $hubFiles = @{ v1_0 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_0.kql') -Raw v1_2 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_2.kql') -Raw - v1_3 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_3.kql') -Raw v1_4 = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_v1_4.kql') -Raw Latest = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_Latest.kql') -Raw } @@ -27,7 +25,7 @@ Describe 'HubsFocusSchemas' { $buildConfig = Get-Content -Path (Join-Path $repoRoot 'src/templates/finops-hub/.build.config') -Raw } - Context 'FOCUS 1.3 columns in Costs_raw' { + Context 'FOCUS 1.4 columns in Costs_raw' { BeforeAll { # Extract just the Costs_raw alter block (not Costs_final or any other table). @@ -47,7 +45,7 @@ Describe 'HubsFocusSchemas' { } } - Context 'ContractCommitment_raw exists with FOCUS 1.3 + 1.4 columns' { + Context 'ContractCommitment_raw exists with all FOCUS 1.4 columns' { BeforeAll { # The Redefine-all-columns alter-table block is the second occurrence; match all and pick it. @@ -63,7 +61,7 @@ Describe 'HubsFocusSchemas' { $contractCommitmentRawBlock | Should -Not -BeNullOrEmpty } - It 'Includes FOCUS 1.3 column <_>' -ForEach @( + It 'Includes base column <_>' -ForEach @( 'BillingCurrency', 'ContractCommitmentCategory', 'ContractCommitmentCost', 'ContractCommitmentId', 'ContractCommitmentPeriodEnd', 'ContractCommitmentPeriodStart', 'ContractCommitmentQuantity', 'ContractCommitmentType', 'ContractCommitmentUnit', @@ -83,47 +81,6 @@ Describe 'HubsFocusSchemas' { } } - Context 'IngestionSetup_v1_3.kql' { - - BeforeAll { - $script:costsFinalV13Block = if ($ingestionFiles.v1_3 -match '(?ms)\.create-merge table Costs_final_v1_3 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } - } - - It 'Defines Costs_transform_v1_3' { - $ingestionFiles.v1_3 | Should -Match 'Costs_transform_v1_3\(\)' - } - - It 'Defines Costs_final_v1_3 table' { - $ingestionFiles.v1_3 | Should -Match '\.create-merge table Costs_final_v1_3' - } - - It 'Defines ContractCommitment_transform_v1_3' { - $ingestionFiles.v1_3 | Should -Match 'ContractCommitment_transform_v1_3\(\)' - } - - It 'Defines ContractCommitment_final_v1_3 table' { - $ingestionFiles.v1_3 | Should -Match '\.create-merge table ContractCommitment_final_v1_3' - } - - It 'Costs_final_v1_3 block was extracted' { - $costsFinalV13Block | Should -Not -BeNullOrEmpty - } - - It 'Costs_final_v1_3 includes FOCUS 1.3 column <_>' -ForEach @( - 'AllocatedMethodId', 'AllocatedMethodDetails', 'AllocatedResourceId', - 'AllocatedResourceName', 'AllocatedTags', 'ContractApplied', - 'ServiceProviderName', 'HostProviderName' - ) { - $costsFinalV13Block | Should -Match "(?m)^\s+$_\s*:" - } - - It 'Costs_final_v1_3 still includes deprecated <_> for back compat' -ForEach @( - 'ProviderName', 'PublisherName' - ) { - $costsFinalV13Block | Should -Match "(?m)^\s+$_\s*:" - } - } - Context 'IngestionSetup_v1_4.kql' { BeforeAll { @@ -139,11 +96,19 @@ Describe 'HubsFocusSchemas' { $ingestionFiles.v1_4 | Should -Match '\.create-merge table Costs_final_v1_4' } + It 'Defines ContractCommitment_transform_v1_4' { + $ingestionFiles.v1_4 | Should -Match 'ContractCommitment_transform_v1_4\(\)' + } + + It 'Defines ContractCommitment_final_v1_4 table' { + $ingestionFiles.v1_4 | Should -Match '\.create-merge table ContractCommitment_final_v1_4' + } + It 'Costs_final_v1_4 block was extracted' { $costsFinalV14Block | Should -Not -BeNullOrEmpty } - It 'Costs_final_v1_4 does NOT include deprecated <_> (removed in 1.4)' -ForEach @( + It 'Costs_final_v1_4 does NOT include deprecated <_> (removed in FOCUS 1.4)' -ForEach @( 'ProviderName', 'PublisherName' ) { $costsFinalV14Block | Should -Not -Match "(?m)^\s+$_\s*:" @@ -159,23 +124,7 @@ Describe 'HubsFocusSchemas' { } } - Context 'HubSetup_v1_3.kql' { - - It 'Defines <_>' -ForEach @( - 'CommitmentDiscountUsage_v1_3', 'ContractCommitment_v1_3', - 'Costs_v1_3', 'Prices_v1_3', 'Recommendations_v1_3', 'Transactions_v1_3' - ) { - $hubFiles.v1_3 | Should -Match "$_\(\)" - } - - It 'Costs_v1_3 unions all prior versions' -ForEach @( - 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_3' - ) { - $hubFiles.v1_3 | Should -Match "database\('Ingestion'\)\.$_" - } - } - - Context 'HubSetup_v1_4.kql (preview)' { + Context 'HubSetup_v1_4.kql' { It 'Defines <_>' -ForEach @( 'CommitmentDiscountUsage_v1_4', 'ContractCommitment_v1_4', @@ -185,42 +134,35 @@ Describe 'HubsFocusSchemas' { } It 'Costs_v1_4 unions all prior versions' -ForEach @( - 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_3', 'Costs_final_v1_4' + 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_4' ) { $hubFiles.v1_4 | Should -Match "database\('Ingestion'\)\.$_" } - - It 'Marks itself as preview in docstring' { - $hubFiles.v1_4 | Should -Match 'FOCUS 1\.4-preview' - } } Context 'HubSetup_Latest.kql aliases' { - It 'Aliases <_> to v1_3 (latest GA, not v1_4 preview)' -ForEach @( - 'CommitmentDiscountUsage', 'Costs', 'Prices', 'Recommendations', 'Transactions', - 'ContractCommitment' + It 'Aliases <_> to v1_4 (latest GA)' -ForEach @( + 'CommitmentDiscountUsage', 'ContractCommitment', 'Costs', 'Prices', 'Recommendations', 'Transactions' ) { - $hubFiles.Latest | Should -Match "(?ms)$_\(\)\s*\{\s*${_}_v1_3\(\)\s*\}" + $hubFiles.Latest | Should -Match "(?ms)$_\(\)\s*\{\s*${_}_v1_4\(\)\s*\}" } - It 'Does NOT alias to v1_4 (preview must be opt-in)' { - $hubFiles.Latest | Should -Not -Match '_v1_4\(\)' + It 'Does NOT alias to v1_2 or older' { + $hubFiles.Latest | Should -Not -Match '_v1_2\(\)' } } Context 'Bicep wiring' { It 'app.bicep loads <_>' -ForEach @( - 'IngestionSetup_v1_3.kql', 'IngestionSetup_v1_4.kql', - 'HubSetup_v1_3.kql', 'HubSetup_v1_4.kql' + 'IngestionSetup_v1_4.kql', 'HubSetup_v1_4.kql' ) { $appBicep | Should -Match ([regex]::Escape($_)) } It '.build.config lists <_>' -ForEach @( - 'IngestionSetup_v1_3.kql', 'IngestionSetup_v1_4.kql', - 'HubSetup_v1_3.kql', 'HubSetup_v1_4.kql' + 'IngestionSetup_v1_4.kql', 'HubSetup_v1_4.kql' ) { $buildConfig | Should -Match ([regex]::Escape($_)) } From a681e73bb920fb783984173903811519c28c823b Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 17 May 2026 20:30:36 +0000 Subject: [PATCH 3/8] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index ceb1b0eba..2b3b5de43 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 05/06/2026 +ms.date: 05/17/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit From 362de4432cc96ed8b9705a607be5757633249631 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 27 May 2026 01:26:45 -0700 Subject: [PATCH 4/8] Cascade PR #2126 feedback to tests + changelog MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Tests: - Pluralize ContractCommitments_raw / _final_v1_4 / _v1_4 assertions - Assert new BillingPeriods_raw + InvoiceDetails_raw datasets exist with their FOCUS 1.4 columns - Assert new BillingPeriods_v1_4(), InvoiceDetails_v1_4(), ContractCommitments_v1_4() hub functions - Assert HubSetup_Latest aliases for BillingPeriods(), ContractCommitments(), InvoiceDetails() - Assert Costs_raw and Costs_final_v1_4 include the 14 new FOCUS 1.4 columns - Assert NO singular ContractCommitment_raw / ContractCommitment_v1_4 / ContractCommitment() exists - Assert Costs_raw keeps removed ProviderName/PublisherName/Region for back compat Changelog: - Expand v15.0.0 entry with the 14 new FOCUS 1.4 columns and the three new supplemental datasets (ContractCommitments, BillingPeriods, InvoiceDetails) - Update plugin entry to list ContractCommitments() / BillingPeriods() / InvoiceDetails() 🤖 Generated with [Claude Code](https://claude.ai/claude-code) Co-Authored-By: Michael Flanakin Co-Authored-By: Claude --- docs-mslearn/toolkit/changelog.md | 6 +- .../Tests/Unit/HubsFocusSchemas.Tests.ps1 | 138 +++++++++++++++--- 2 files changed, 120 insertions(+), 24 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 2b3b5de43..0336977cb 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -32,13 +32,13 @@ The following section lists features and enhancements that are currently in deve - Added 4 agents (CFO, FinOps practitioner, database query, hubs agent), 5 commands (`/ftk-hubs-connect`, `/ftk-hubs-healthCheck`, `/ftk-mom-report`, `/ftk-ytd-report`, `/ftk-cost-optimization`), and an output style. - Linked to the existing KQL query catalog in `src/queries/` from the plugin. - **Changed** - - Updated plugin skill files to reflect FOCUS 1.4 hub schema, the new `ContractCommitment()` function, and the removal of `ProviderName` / `PublisherName` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + - Updated plugin skill files to reflect FOCUS 1.4 hub schema, the new `ContractCommitments()`, `BillingPeriods()`, and `InvoiceDetails()` functions, and the removal of `ProviderName` / `PublisherName` ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). ### FinOps hubs v15.0.0 - **Added** - - Added FOCUS 1.4 hub schema (`v1_4`). New cost and usage columns: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`. Removes deprecated `ProviderName` and `PublisherName`. New supplemental dataset: `ContractCommitment` with 28 columns ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). - - Added unversioned `ContractCommitment()` function aliasing to `ContractCommitment_v1_4()`. + - Added FOCUS 1.4 hub schema (`v1_4`). New cost and usage columns: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`, `CommitmentProgramEligibilityDetails`, `InvoiceDetailId`, and 12 `ContractCommitment*` per-row columns (`ContractCommitmentBenefitCategory`, `ContractCommitmentCreated`, `ContractCommitmentDiscountPercentage`, `ContractCommitmentDurationType`, `ContractCommitmentFulfillmentInterval`, `ContractCommitmentLastUpdated`, `ContractCommitmentLifecycleStatus`, `ContractCommitmentModel`, `ContractCommitmentOfferCategory`, `ContractCommitmentPaymentInterval`, `ContractCommitmentPaymentModel`, `ContractCommitmentPaymentUpfrontPercentage`). Removes deprecated `ProviderName` and `PublisherName` from the schema (raw tables keep them for back compat). Three new supplemental datasets: `ContractCommitments` (28 columns), `BillingPeriods` (6 columns), and `InvoiceDetails` (22 columns) ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + - Added unversioned `ContractCommitments()`, `BillingPeriods()`, and `InvoiceDetails()` functions aliasing to their `_v1_4` counterparts. - **Changed** - Retargeted unversioned `Costs()`, `Prices()`, `CommitmentDiscountUsage()`, `Recommendations()`, and `Transactions()` aliases to their `_v1_4` counterparts. - Refreshed the canonical "add a new FOCUS version" procedure in [src/templates/finops-hub/docs/README.md](https://github.com/microsoft/finops-toolkit/tree/dev/src/templates/finops-hub/docs/README.md) with multi-version-cycle, plugin, and changelog steps. diff --git a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 index f49ff5483..02f4d3584 100644 --- a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 +++ b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 @@ -39,26 +39,43 @@ Describe 'HubsFocusSchemas' { It 'Adds <_> to Costs_raw' -ForEach @( 'AllocatedMethodId', 'AllocatedMethodDetails', 'AllocatedResourceId', 'AllocatedResourceName', 'AllocatedTags', 'ContractApplied', - 'ServiceProviderName', 'HostProviderName' + 'ServiceProviderName', 'HostProviderName', + 'CommitmentProgramEligibilityDetails', 'InvoiceDetailId', + 'ContractCommitmentBenefitCategory', 'ContractCommitmentCreated', + 'ContractCommitmentDiscountPercentage', 'ContractCommitmentDurationType', + 'ContractCommitmentFulfillmentInterval', 'ContractCommitmentLastUpdated', + 'ContractCommitmentLifecycleStatus', 'ContractCommitmentModel', + 'ContractCommitmentOfferCategory', 'ContractCommitmentPaymentInterval', + 'ContractCommitmentPaymentModel', 'ContractCommitmentPaymentUpfrontPercentage' + ) { + $costsRawBlock | Should -Match "(?m)^\s+$_\s*:" + } + + It 'Keeps deprecated <_> for back compat' -ForEach @( + 'ProviderName', 'PublisherName', 'Region' ) { $costsRawBlock | Should -Match "(?m)^\s+$_\s*:" } } - Context 'ContractCommitment_raw exists with all FOCUS 1.4 columns' { + Context 'ContractCommitments_raw exists with all FOCUS 1.4 columns' { BeforeAll { # The Redefine-all-columns alter-table block is the second occurrence; match all and pick it. - $allBlocks = [regex]::Matches($rawTablesContent, '(?ms)\.alter table ContractCommitment_raw \(\r?\n(.*?)\r?\n\)') - $script:contractCommitmentRawBlock = if ($allBlocks.Count -ge 1) { $allBlocks[$allBlocks.Count - 1].Groups[1].Value } else { '' } + $allBlocks = [regex]::Matches($rawTablesContent, '(?ms)\.alter table ContractCommitments_raw \(\r?\n(.*?)\r?\n\)') + $script:contractCommitmentsRawBlock = if ($allBlocks.Count -ge 1) { $allBlocks[$allBlocks.Count - 1].Groups[1].Value } else { '' } + } + + It 'Defines ContractCommitments_raw (plural)' { + $rawTablesContent | Should -Match '\.alter table ContractCommitments_raw \(' } - It 'Defines ContractCommitment_raw' { - $rawTablesContent | Should -Match '\.alter table ContractCommitment_raw \(' + It 'Does NOT define singular ContractCommitment_raw' { + $rawTablesContent | Should -Not -Match '\.alter table ContractCommitment_raw \(' } - It 'ContractCommitment_raw column block was extracted' { - $contractCommitmentRawBlock | Should -Not -BeNullOrEmpty + It 'ContractCommitments_raw column block was extracted' { + $contractCommitmentsRawBlock | Should -Not -BeNullOrEmpty } It 'Includes base column <_>' -ForEach @( @@ -68,7 +85,7 @@ Describe 'HubsFocusSchemas' { 'ContractId', 'ContractPeriodEnd', 'ContractPeriodStart', 'InvoiceIssuerName', 'PricingCurrency' ) { - $contractCommitmentRawBlock | Should -Match "(?m)^\s+$_\s*:" + $contractCommitmentsRawBlock | Should -Match "(?m)^\s+$_\s*:" } It 'Includes FOCUS 1.4 column <_>' -ForEach @( @@ -77,7 +94,49 @@ Describe 'HubsFocusSchemas' { 'LifecycleStatus', 'Model', 'OfferCategory', 'PaymentInterval', 'PaymentModel', 'PaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost' ) { - $contractCommitmentRawBlock | Should -Match "(?m)^\s+$_\s*:" + $contractCommitmentsRawBlock | Should -Match "(?m)^\s+$_\s*:" + } + } + + Context 'BillingPeriods_raw exists with FOCUS 1.4 columns' { + + BeforeAll { + $allBlocks = [regex]::Matches($rawTablesContent, '(?ms)\.alter table BillingPeriods_raw \(\r?\n(.*?)\r?\n\)') + $script:billingPeriodsRawBlock = if ($allBlocks.Count -ge 1) { $allBlocks[$allBlocks.Count - 1].Groups[1].Value } else { '' } + } + + It 'Defines BillingPeriods_raw' { + $rawTablesContent | Should -Match '\.alter table BillingPeriods_raw \(' + } + + It 'Includes column <_>' -ForEach @( + 'BillingPeriodCreated', 'BillingPeriodEnd', 'BillingPeriodLastUpdated', + 'BillingPeriodStart', 'BillingPeriodStatus', 'InvoiceIssuerName' + ) { + $billingPeriodsRawBlock | Should -Match "(?m)^\s+$_\s*:" + } + } + + Context 'InvoiceDetails_raw exists with FOCUS 1.4 columns' { + + BeforeAll { + $allBlocks = [regex]::Matches($rawTablesContent, '(?ms)\.alter table InvoiceDetails_raw \(\r?\n(.*?)\r?\n\)') + $script:invoiceDetailsRawBlock = if ($allBlocks.Count -ge 1) { $allBlocks[$allBlocks.Count - 1].Groups[1].Value } else { '' } + } + + It 'Defines InvoiceDetails_raw' { + $rawTablesContent | Should -Match '\.alter table InvoiceDetails_raw \(' + } + + It 'Includes column <_>' -ForEach @( + 'BilledCost', 'BillingAccountId', 'BillingCurrency', 'BillingPeriodEnd', + 'BillingPeriodStart', 'ChargeCategory', 'InvoiceDetailCreated', 'InvoiceDetailDescription', + 'InvoiceDetailGrain', 'InvoiceDetailId', 'InvoiceDetailLastUpdated', 'InvoiceId', + 'InvoiceIssueDate', 'InvoiceIssueStatus', 'InvoiceIssuerName', 'PaymentCurrency', + 'PaymentCurrencyBilledCost', 'PaymentCurrencyInvoiceDetailId', 'PaymentDueDate', + 'PaymentTerms', 'PurchaseOrderNumber', 'ReferenceInvoiceId' + ) { + $invoiceDetailsRawBlock | Should -Match "(?m)^\s+$_\s*:" } } @@ -85,7 +144,7 @@ Describe 'HubsFocusSchemas' { BeforeAll { $script:costsFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table Costs_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } - $script:contractCommitmentFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table ContractCommitment_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } + $script:contractCommitmentsFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table ContractCommitments_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } } It 'Defines Costs_transform_v1_4' { @@ -96,43 +155,75 @@ Describe 'HubsFocusSchemas' { $ingestionFiles.v1_4 | Should -Match '\.create-merge table Costs_final_v1_4' } - It 'Defines ContractCommitment_transform_v1_4' { - $ingestionFiles.v1_4 | Should -Match 'ContractCommitment_transform_v1_4\(\)' + It 'Defines ContractCommitments_transform_v1_4 (plural)' { + $ingestionFiles.v1_4 | Should -Match 'ContractCommitments_transform_v1_4\(\)' + } + + It 'Defines ContractCommitments_final_v1_4 table (plural)' { + $ingestionFiles.v1_4 | Should -Match '\.create-merge table ContractCommitments_final_v1_4' + } + + It 'Defines BillingPeriods_transform_v1_4' { + $ingestionFiles.v1_4 | Should -Match 'BillingPeriods_transform_v1_4\(\)' + } + + It 'Defines BillingPeriods_final_v1_4 table' { + $ingestionFiles.v1_4 | Should -Match '\.create-merge table BillingPeriods_final_v1_4' + } + + It 'Defines InvoiceDetails_transform_v1_4' { + $ingestionFiles.v1_4 | Should -Match 'InvoiceDetails_transform_v1_4\(\)' } - It 'Defines ContractCommitment_final_v1_4 table' { - $ingestionFiles.v1_4 | Should -Match '\.create-merge table ContractCommitment_final_v1_4' + It 'Defines InvoiceDetails_final_v1_4 table' { + $ingestionFiles.v1_4 | Should -Match '\.create-merge table InvoiceDetails_final_v1_4' } It 'Costs_final_v1_4 block was extracted' { $costsFinalV14Block | Should -Not -BeNullOrEmpty } - It 'Costs_final_v1_4 does NOT include deprecated <_> (removed in FOCUS 1.4)' -ForEach @( + It 'Costs_final_v1_4 does NOT include removed <_>' -ForEach @( 'ProviderName', 'PublisherName' ) { $costsFinalV14Block | Should -Not -Match "(?m)^\s+$_\s*:" } - It 'ContractCommitment_final_v1_4 includes FOCUS 1.4 column <_>' -ForEach @( + It 'Costs_final_v1_4 includes new FOCUS 1.4 column <_>' -ForEach @( + 'CommitmentProgramEligibilityDetails', 'InvoiceDetailId', + 'ContractCommitmentBenefitCategory', 'ContractCommitmentCreated', + 'ContractCommitmentDiscountPercentage', 'ContractCommitmentDurationType', + 'ContractCommitmentFulfillmentInterval', 'ContractCommitmentLastUpdated', + 'ContractCommitmentLifecycleStatus', 'ContractCommitmentModel', + 'ContractCommitmentOfferCategory', 'ContractCommitmentPaymentInterval', + 'ContractCommitmentPaymentModel', 'ContractCommitmentPaymentUpfrontPercentage' + ) { + $costsFinalV14Block | Should -Match "(?m)^\s+$_\s*:" + } + + It 'ContractCommitments_final_v1_4 includes FOCUS 1.4 column <_>' -ForEach @( 'BenefitCategory', 'ContractCommitmentApplicability', 'Created', 'DiscountPercentage', 'DurationType', 'FulfillmentInterval', 'LastUpdated', 'LifecycleStatus', 'Model', 'OfferCategory', 'PaymentInterval', 'PaymentModel', 'PaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost' ) { - $contractCommitmentFinalV14Block | Should -Match "(?m)^\s+$_\s*:" + $contractCommitmentsFinalV14Block | Should -Match "(?m)^\s+$_\s*:" } } Context 'HubSetup_v1_4.kql' { It 'Defines <_>' -ForEach @( - 'CommitmentDiscountUsage_v1_4', 'ContractCommitment_v1_4', - 'Costs_v1_4', 'Prices_v1_4', 'Recommendations_v1_4', 'Transactions_v1_4' + 'BillingPeriods_v1_4', 'CommitmentDiscountUsage_v1_4', 'ContractCommitments_v1_4', + 'Costs_v1_4', 'InvoiceDetails_v1_4', 'Prices_v1_4', 'Recommendations_v1_4', 'Transactions_v1_4' ) { $hubFiles.v1_4 | Should -Match "$_\(\)" } + It 'Does NOT define singular ContractCommitment_v1_4' { + $hubFiles.v1_4 | Should -Not -Match 'ContractCommitment_v1_4\(\)' + } + It 'Costs_v1_4 unions all prior versions' -ForEach @( 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_4' ) { @@ -143,11 +234,16 @@ Describe 'HubsFocusSchemas' { Context 'HubSetup_Latest.kql aliases' { It 'Aliases <_> to v1_4 (latest GA)' -ForEach @( - 'CommitmentDiscountUsage', 'ContractCommitment', 'Costs', 'Prices', 'Recommendations', 'Transactions' + 'BillingPeriods', 'CommitmentDiscountUsage', 'ContractCommitments', + 'Costs', 'InvoiceDetails', 'Prices', 'Recommendations', 'Transactions' ) { $hubFiles.Latest | Should -Match "(?ms)$_\(\)\s*\{\s*${_}_v1_4\(\)\s*\}" } + It 'Does NOT alias to singular ContractCommitment' { + $hubFiles.Latest | Should -Not -Match '(?m)^ContractCommitment\(\)' + } + It 'Does NOT alias to v1_2 or older' { $hubFiles.Latest | Should -Not -Match '_v1_2\(\)' } From bbb71802cb3849363c16e7b8883a212397c255ee Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Wed, 27 May 2026 08:28:04 +0000 Subject: [PATCH 5/8] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 0336977cb..4c50c8c1d 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 05/17/2026 +ms.date: 05/27/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit From 9eb79cadc0e96ebf4f2991b49c3385d7607ced71 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 15 Jul 2026 13:47:59 -0700 Subject: [PATCH 6/8] test(hubs): rework FOCUS schema tests into a glob-based regression harness Rework HubsFocusSchemas.Tests.ps1 per FOCUS-1.4-PLAN.md D6: - Check 1: glob-discover all IngestionSetup_v1_*/HubSetup_v1_* scripts and assert .build.config + app.bicep registration (covers the D9 split file and future versions automatically; prevents regression #1777) - Check 2 (new): version-string consistency with a data-driven allowlist (union arms, deprecation docstrings, known benign TODO) plus v1_3 and git conflict marker guards - Check 3 (new): exactly one enabled update policy schema version, derived from the highest discovered version; every raw source stays wired - Check 4: tightened union arm occurrence counts (5 managed datasets x 3 versions + 3 new v1_4-only datasets), back-compat _final_v1_4 arms in v1_0/v1_2 hub functions, _v1_0()/_v1_2() alias guards in HubSetup_Latest, and the D3 provider column rename round trip with an inversion guard Also reconcile the v15 changelog with what the stack ships: FOCUS 1.4 conversion behavior, three new datasets (empty until Cost Management exports FOCUS 1.4), v1_2 deprecation/policy disablement, provider column removal with down-conversion, and the dashboard repoint. All new checks were red-green validated against injected defects. Co-Authored-By: Claude Fable 5 --- docs-mslearn/toolkit/changelog.md | 19 +- .../Tests/Unit/HubsFocusSchemas.Tests.ps1 | 351 +++++++++++++++--- 2 files changed, 309 insertions(+), 61 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 4c50c8c1d..e260b4f04 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 05/27/2026 +ms.date: 07/15/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit @@ -37,11 +37,22 @@ The following section lists features and enhancements that are currently in deve ### FinOps hubs v15.0.0 - **Added** - - Added FOCUS 1.4 hub schema (`v1_4`). New cost and usage columns: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`, `CommitmentProgramEligibilityDetails`, `InvoiceDetailId`, and 12 `ContractCommitment*` per-row columns (`ContractCommitmentBenefitCategory`, `ContractCommitmentCreated`, `ContractCommitmentDiscountPercentage`, `ContractCommitmentDurationType`, `ContractCommitmentFulfillmentInterval`, `ContractCommitmentLastUpdated`, `ContractCommitmentLifecycleStatus`, `ContractCommitmentModel`, `ContractCommitmentOfferCategory`, `ContractCommitmentPaymentInterval`, `ContractCommitmentPaymentModel`, `ContractCommitmentPaymentUpfrontPercentage`). Removes deprecated `ProviderName` and `PublisherName` from the schema (raw tables keep them for back compat). Three new supplemental datasets: `ContractCommitments` (28 columns), `BillingPeriods` (6 columns), and `InvoiceDetails` (22 columns) ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). - - Added unversioned `ContractCommitments()`, `BillingPeriods()`, and `InvoiceDetails()` functions aliasing to their `_v1_4` counterparts. + - Added full support for FOCUS 1.4 (`v1_4` schema) in Azure Data Explorer and Microsoft Fabric across all managed datasets ([#2120](https://github.com/microsoft/finops-toolkit/issues/2120)). + - When FOCUS 1.0 or 1.2 data is ingested, it's converted to FOCUS 1.4. Historical data remains in the `*_final_v1_0` and `*_final_v1_2` tables. + - New versioned Hub database functions, like `Costs_v1_4()`, were added based on the FOCUS 1.4 schema changes. + - Make sure you use versioned functions, like `Costs_v1_2()`, to avoid breaking changes between FOCUS versions. + - Added new columns to the Costs managed dataset: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`, `CommitmentProgramEligibilityDetails`, `InvoiceDetailId`, and 12 `ContractCommitment*` per-row columns (`ContractCommitmentBenefitCategory`, `ContractCommitmentCreated`, `ContractCommitmentDiscountPercentage`, `ContractCommitmentDurationType`, `ContractCommitmentFulfillmentInterval`, `ContractCommitmentLastUpdated`, `ContractCommitmentLifecycleStatus`, `ContractCommitmentModel`, `ContractCommitmentOfferCategory`, `ContractCommitmentPaymentInterval`, `ContractCommitmentPaymentModel`, `ContractCommitmentPaymentUpfrontPercentage`). + - Added three new FOCUS 1.4 datasets: `ContractCommitments` (28 columns), `BillingPeriods` (6 columns), and `InvoiceDetails` (22 columns), each with tables and versioned and unversioned functions (for example, `ContractCommitments()` and `ContractCommitments_v1_4()`). + - These datasets remain empty until Cost Management supports exporting FOCUS 1.4 data. + - Added a static test harness for the hub database setup scripts that verifies build registration, schema version consistency, and update policy state to catch regressions before release. - **Changed** - Retargeted unversioned `Costs()`, `Prices()`, `CommitmentDiscountUsage()`, `Recommendations()`, and `Transactions()` aliases to their `_v1_4` counterparts. - - Refreshed the canonical "add a new FOCUS version" procedure in [src/templates/finops-hub/docs/README.md](https://github.com/microsoft/finops-toolkit/tree/dev/src/templates/finops-hub/docs/README.md) with multi-version-cycle, plugin, and changelog steps. + - This is **not** a breaking change if you are following the prescribed guidance of using versioned functions. + - Updated the Data Explorer dashboard to use the `v1_4` schema. +- **Deprecated** + - Deprecated the FOCUS 1.2 transform functions and disabled the `v1_2` update policies. New data is transformed to FOCUS 1.4; data ingested under older versions remains available through versioned functions like `Costs_v1_2()`. +- **Removed** + - Removed deprecated `ProviderName` and `PublisherName` columns from the `v1_4` schema in favor of the FOCUS 1.3 replacements, `HostProviderName` and `ServiceProviderName`. The `v1_0` and `v1_2` functions still return both columns by down-converting from the new columns, and raw tables keep them for back compat. ### Bicep Registry module pending updates diff --git a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 index 02f4d3584..599c2c662 100644 --- a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 +++ b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 @@ -1,17 +1,66 @@ # Copyright (c) Microsoft Corporation. # Licensed under the MIT License. +<# + Static regression harness for the FinOps hub FOCUS schema setup scripts (Analytics/scripts/*.kql). + + Covers the four D6 check categories from the FOCUS 1.4 plan, each mapped to a v1_2-era regression: + 1. Registration completeness - every versioned setup script is registered in .build.config (Fabric + bundle) and loaded in Analytics/app.bicep (prevents #1777: v1_2 script missing from Fabric build). + 2. Version-string consistency - versioned files only reference other schema versions through + allowlisted patterns, and no conflict markers or phantom versions exist (prevents stale + cross-version copy/paste references and committed merge conflict markers). + 3. Exactly one enabled update policy version - all enabled update policies live in the current + (highest) schema version; older versions are fully disabled (prevents double ingestion). + 4. Column and conversion contracts - v1_4 tables carry the FOCUS 1.4 columns, hub functions union + every schema version, and the provider column rename round-trips without inversion. + + All checks are static text analysis; no Azure Data Explorer connection is needed. +#> + Describe 'HubsFocusSchemas' { - BeforeAll { + BeforeDiscovery { $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path $scriptsPath = Join-Path $repoRoot 'src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/scripts' + + # Discover versioned setup scripts dynamically so new files (e.g., a future v1_5 or another + # size-limit split file) are covered without updating this test. + $ingestionSetupFiles = @(Get-ChildItem -Path $scriptsPath -Filter 'IngestionSetup_v1_*.kql' | ForEach-Object { + @{ Name = $_.Name; FullName = $_.FullName; Version = $(if ($_.Name -match '_v1_(\d+)') { $Matches[1] }) } + }) + $hubSetupFiles = @(Get-ChildItem -Path $scriptsPath -Filter 'HubSetup_v1_*.kql' | ForEach-Object { + @{ Name = $_.Name; FullName = $_.FullName; Version = $(if ($_.Name -match '_v1_(\d+)') { $Matches[1] }) } + }) + $versionedSetupFiles = $ingestionSetupFiles + $hubSetupFiles + $registeredSetupFiles = $versionedSetupFiles + @(Get-ChildItem -Path $scriptsPath -Filter 'HubSetup_Latest.kql' | ForEach-Object { + @{ Name = $_.Name; FullName = $_.FullName; Version = $null } + }) + + # Files that define the FOCUS 1.4 side of the provider column rename (up-conversion). + $v14SetupFiles = @($versionedSetupFiles | Where-Object { $_.Version -eq '4' }) + + # Managed datasets that existed before FOCUS 1.4 (unioned across v1_0, v1_2, and v1_4). + $managedDatasets = @('CommitmentDiscountUsage', 'Costs', 'Prices', 'Recommendations', 'Transactions') + + # Supplemental datasets introduced with FOCUS 1.4 (v1_4 final tables only). + $v14OnlyDatasets = @('BillingPeriods', 'ContractCommitments', 'InvoiceDetails') + + $allDatasets = @($managedDatasets + $v14OnlyDatasets | Sort-Object) + } + + BeforeAll { + $repoRoot = (Resolve-Path "$PSScriptRoot/../../../..").Path + $hubRoot = Join-Path $repoRoot 'src/templates/finops-hub' + $scriptsPath = Join-Path $hubRoot 'modules/Microsoft.FinOpsHubs/Analytics/scripts' $rawTablesContent = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_RawTables.kql') -Raw $ingestionFiles = @{ v1_0 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_0.kql') -Raw v1_2 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_2.kql') -Raw - v1_4 = Get-Content -Path (Join-Path $scriptsPath 'IngestionSetup_v1_4.kql') -Raw + # FOCUS 1.4 ingestion setup is split across two files to stay under the Bicep + # loadTextContent() 131 KB limit (D9); combine them for schema-level assertions. + v1_4 = @(Get-ChildItem -Path $scriptsPath -Filter 'IngestionSetup_v1_4*.kql' | ForEach-Object { Get-Content -Path $_.FullName -Raw }) -join "`n" } $hubFiles = @{ @@ -21,8 +70,185 @@ Describe 'HubsFocusSchemas' { Latest = Get-Content -Path (Join-Path $scriptsPath 'HubSetup_Latest.kql') -Raw } - $appBicep = Get-Content -Path (Join-Path $repoRoot 'src/templates/finops-hub/modules/Microsoft.FinOpsHubs/Analytics/app.bicep') -Raw - $buildConfig = Get-Content -Path (Join-Path $repoRoot 'src/templates/finops-hub/.build.config') -Raw + $appBicep = Get-Content -Path (Join-Path $hubRoot 'modules/Microsoft.FinOpsHubs/Analytics/app.bicep') -Raw + $buildConfig = Get-Content -Path (Join-Path $hubRoot '.build.config') -Raw | ConvertFrom-Json + $combineKqlFiles = @($buildConfig.combineKql | ForEach-Object { $_.files }) + + # Allowlist of legitimate cross-version reference patterns. Any other reference to a schema + # version different from the file's own version fails the version-string consistency check. + # Add new entries deliberately - each one is an explicit exception, not a convenience. + $crossVersionAllowlist = @( + @{ + Pattern = "database\('Ingestion'\)\.\w+_final_v1_\d" + Reason = 'Hub setup functions union final tables from every schema version by design.' + } + @{ + Pattern = "docstring\s*=?\s*'DEPRECATED:.*Use \w+_v1_\d\(\) instead" + Reason = 'Deprecation docstrings point to the successor version function.' + } + @{ + Pattern = 'TODO: Remove x_SourceChanges in v1_3 \(or later\)' + Reason = 'Known benign TODO in IngestionSetup_v1_2.kql; v1_3 was skipped as a hub schema version.' + } + ) + + # Returns cross-version reference violations for one file: every identifier ending in _v1_ + # where N is not the file's own version and the line matches no allowlist pattern. + function Get-CrossVersionViolation([string]$FilePath, [string]$OwnVersion) + { + $violations = @() + $lineNumber = 0 + foreach ($line in [System.IO.File]::ReadAllLines($FilePath)) + { + $lineNumber++ + foreach ($token in [regex]::Matches($line, '[A-Za-z0-9]\w*_v1_(\d+)')) + { + if ($token.Groups[1].Value -eq $OwnVersion) { continue } + $isAllowed = $false + foreach ($rule in $crossVersionAllowlist) + { + if ($line -match $rule.Pattern) { $isAllowed = $true; break } + } + if (-not $isAllowed) + { + $violations += "$(Split-Path -Leaf $FilePath):${lineNumber}: '$($token.Value)' -- $($line.Trim())" + } + } + } + return $violations + } + + # Parse every update policy from the ingestion setup scripts: + # ".alter table policy update" followed by a fenced JSON array. + $updatePolicies = @() + foreach ($file in (Get-ChildItem -Path $scriptsPath -Filter 'IngestionSetup_v1_*.kql')) + { + $fileVersion = if ($file.Name -match '_v1_(\d+)') { [int]$Matches[1] } else { $null } + $content = Get-Content -Path $file.FullName -Raw + foreach ($match in [regex]::Matches($content, '(?ms)^\.alter table (?\S+) policy update[ \t]*\r?\n```[ \t]*\r?\n(?.*?)```')) + { + foreach ($policy in ($match.Groups['json'].Value | ConvertFrom-Json)) + { + $updatePolicies += @{ + File = $file.Name + Version = $fileVersion + Target = $match.Groups['target'].Value + Source = $policy.Source + IsEnabled = $policy.IsEnabled + } + } + } + } + + # Derive the current schema version instead of hardcoding it so this test survives v1_5. + $currentSchemaVersion = ($updatePolicies | ForEach-Object { $_.Version } | Measure-Object -Maximum).Maximum + } + + Context 'Registration completeness (D6 check 1)' { + + It 'Discovers versioned setup scripts via glob' { + # Guard against a broken glob silently green-washing the -ForEach tests below. + @(Get-ChildItem -Path $scriptsPath -Filter 'IngestionSetup_v1_*.kql').Count | Should -BeGreaterOrEqual 3 + @(Get-ChildItem -Path $scriptsPath -Filter 'HubSetup_v1_*.kql').Count | Should -BeGreaterOrEqual 3 + } + + It ' is registered in .build.config combineKql' -ForEach $registeredSetupFiles { + $combineKqlFiles | Should -Contain "modules/Microsoft.FinOpsHubs/Analytics/scripts/$Name" -Because "every setup script must ship in the Fabric KQL bundle (regression #1777: IngestionSetup_v1_2.kql was missing from .build.config)" + } + + It ' is loaded via loadTextContent in app.bicep' -ForEach $registeredSetupFiles { + $appBicep | Should -Match ([regex]::Escape("loadTextContent('scripts/$Name')")) -Because "every setup script must be embedded in the Data Explorer deployment" + } + + It 'Lists versioned HubSetup files before HubSetup_Latest.kql in .build.config' { + $hubBundle = @($buildConfig.combineKql | Where-Object { $_.files -match 'HubSetup_Latest\.kql' }) + $hubBundle.Count | Should -Be 1 + $files = @($hubBundle[0].files) + $latestIndex = $files.IndexOf('modules/Microsoft.FinOpsHubs/Analytics/scripts/HubSetup_Latest.kql') + $latestIndex | Should -BeGreaterOrEqual 0 + foreach ($file in ($files -match 'HubSetup_v1_\d+\.kql$')) + { + $files.IndexOf($file) | Should -BeLessThan $latestIndex -Because "unversioned aliases in HubSetup_Latest.kql depend on the versioned functions being created first" + } + } + } + + Context 'Version-string consistency (D6 check 2)' { + + It ' only references other schema versions via allowlisted patterns' -ForEach $versionedSetupFiles { + $violations = Get-CrossVersionViolation -FilePath $FullName -OwnVersion $Version + ($violations -join "`n") | Should -BeNullOrEmpty -Because 'cross-version identifiers outside the allowlist are usually copy/paste defects from cloning the previous version file' + } + + It 'Does not reference v1_3 anywhere in Analytics scripts' { + # v1_3 was never a hub schema version; any reference is a typo or bad copy/paste. + $violations = @() + foreach ($file in (Get-ChildItem -Path $scriptsPath -Filter '*.kql')) + { + $lineNumber = 0 + foreach ($line in [System.IO.File]::ReadAllLines($file.FullName)) + { + $lineNumber++ + if ($line -notmatch 'v1_3') { continue } + $isAllowed = $false + foreach ($rule in $crossVersionAllowlist) + { + if ($line -match $rule.Pattern) { $isAllowed = $true; break } + } + if (-not $isAllowed) + { + $violations += "$($file.Name):${lineNumber}: $($line.Trim())" + } + } + } + ($violations -join "`n") | Should -BeNullOrEmpty + } + + It 'Has no git conflict markers in finops-hub KQL or Bicep files' { + $violations = @() + foreach ($file in (Get-ChildItem -Path $hubRoot -Recurse -Include '*.kql', '*.bicep')) + { + $lineNumber = 0 + foreach ($line in [System.IO.File]::ReadAllLines($file.FullName)) + { + $lineNumber++ + if ($line -match '^(<{7} |={7}$|>{7} )') + { + $violations += "$($file.Name):${lineNumber}: $($line.Trim())" + } + } + } + ($violations -join "`n") | Should -BeNullOrEmpty -Because 'conflict markers have been committed to these files before; they break the ADX database script' + } + } + + Context 'Update policies: exactly one enabled schema version (D6 check 3)' { + + It 'Parses update policies from ' -ForEach $ingestionSetupFiles { + @($updatePolicies | Where-Object { $_.File -eq $Name }).Count | Should -BeGreaterThan 0 -Because 'every ingestion setup script defines update policies; zero parsed policies means the parser regex no longer matches the file format' + } + + It 'Enables update policies in exactly one schema version (the current one)' { + $enabledVersions = @($updatePolicies | Where-Object { $_.IsEnabled } | ForEach-Object { $_.Version } | Sort-Object -Unique) + $enabledVersions | Should -Be @($currentSchemaVersion) -Because 'enabled update policies in more than one schema version cause double ingestion; older versions must be fully disabled when a new version ships' + } + + It ' has zero enabled update policies (superseded version)' -ForEach ($ingestionSetupFiles | Where-Object { [int]$_.Version -lt (@($ingestionSetupFiles | ForEach-Object { [int]$_.Version }) | Measure-Object -Maximum).Maximum }) { + $enabled = @($updatePolicies | Where-Object { $_.File -eq $Name -and $_.IsEnabled }) + ($enabled | ForEach-Object { "$($_.Source) -> $($_.Target)" }) -join "`n" | Should -BeNullOrEmpty + } + + It 'Every raw source table has an enabled update policy in the current schema version' { + # Union of Source tables across all versions = every raw table that ever fed a transform. + # Each must still be wired to the current version (counts are flexible: Costs_raw is fed by + # ActualCosts_raw and AmortizedCosts_raw in addition to its own transform, per C360 support). + $allSources = @($updatePolicies | ForEach-Object { $_.Source } | Sort-Object -Unique) + $enabledSources = @($updatePolicies | Where-Object { $_.IsEnabled -and $_.Version -eq $currentSchemaVersion } | ForEach-Object { $_.Source } | Sort-Object -Unique) + foreach ($source in $allSources) + { + $enabledSources | Should -Contain $source -Because "data ingested into $source would silently stop flowing without an enabled update policy in the current schema version" + } + } } Context 'FOCUS 1.4 columns in Costs_raw' { @@ -140,43 +366,27 @@ Describe 'HubsFocusSchemas' { } } - Context 'IngestionSetup_v1_4.kql' { + Context 'IngestionSetup v1_4 transforms and final tables' { BeforeAll { $script:costsFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table Costs_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } $script:contractCommitmentsFinalV14Block = if ($ingestionFiles.v1_4 -match '(?ms)\.create-merge table ContractCommitments_final_v1_4 \(\r?\n(.*?)\r?\n\)') { $Matches[1] } else { '' } } - It 'Defines Costs_transform_v1_4' { - $ingestionFiles.v1_4 | Should -Match 'Costs_transform_v1_4\(\)' - } - - It 'Defines Costs_final_v1_4 table' { - $ingestionFiles.v1_4 | Should -Match '\.create-merge table Costs_final_v1_4' - } - - It 'Defines ContractCommitments_transform_v1_4 (plural)' { - $ingestionFiles.v1_4 | Should -Match 'ContractCommitments_transform_v1_4\(\)' - } - - It 'Defines ContractCommitments_final_v1_4 table (plural)' { - $ingestionFiles.v1_4 | Should -Match '\.create-merge table ContractCommitments_final_v1_4' - } - - It 'Defines BillingPeriods_transform_v1_4' { - $ingestionFiles.v1_4 | Should -Match 'BillingPeriods_transform_v1_4\(\)' - } - - It 'Defines BillingPeriods_final_v1_4 table' { - $ingestionFiles.v1_4 | Should -Match '\.create-merge table BillingPeriods_final_v1_4' + It 'Defines <_>_transform_v1_4()' -ForEach @( + 'BillingPeriods', 'ContractCommitments', 'Costs', 'InvoiceDetails', 'Prices' + ) { + $ingestionFiles.v1_4 | Should -Match "$($_)_transform_v1_4\(\)" } - It 'Defines InvoiceDetails_transform_v1_4' { - $ingestionFiles.v1_4 | Should -Match 'InvoiceDetails_transform_v1_4\(\)' + It 'Defines <_>_final_v1_4 table' -ForEach @( + 'BillingPeriods', 'ContractCommitments', 'Costs', 'InvoiceDetails', 'Prices' + ) { + $ingestionFiles.v1_4 | Should -Match "\.create-merge table $($_)_final_v1_4" } - It 'Defines InvoiceDetails_final_v1_4 table' { - $ingestionFiles.v1_4 | Should -Match '\.create-merge table InvoiceDetails_final_v1_4' + It 'Does NOT define singular ContractCommitment_final_v1_4' { + $ingestionFiles.v1_4 | Should -Not -Match '\.create-merge table ContractCommitment_final_v1_4' } It 'Costs_final_v1_4 block was extracted' { @@ -213,31 +423,68 @@ Describe 'HubsFocusSchemas' { Context 'HubSetup_v1_4.kql' { - It 'Defines <_>' -ForEach @( - 'BillingPeriods_v1_4', 'CommitmentDiscountUsage_v1_4', 'ContractCommitments_v1_4', - 'Costs_v1_4', 'InvoiceDetails_v1_4', 'Prices_v1_4', 'Recommendations_v1_4', 'Transactions_v1_4' - ) { - $hubFiles.v1_4 | Should -Match "$_\(\)" + It 'Defines <_>_v1_4()' -ForEach $allDatasets { + $hubFiles.v1_4 | Should -Match "$($_)_v1_4\(\)" } It 'Does NOT define singular ContractCommitment_v1_4' { $hubFiles.v1_4 | Should -Not -Match 'ContractCommitment_v1_4\(\)' } - It 'Costs_v1_4 unions all prior versions' -ForEach @( - 'Costs_final_v1_0', 'Costs_final_v1_2', 'Costs_final_v1_4' + It 'Unions <_> final tables from all three schema versions exactly once each' -ForEach $managedDatasets { + foreach ($version in @('v1_0', 'v1_2', 'v1_4')) + { + [regex]::Matches($hubFiles.v1_4, "database\('Ingestion'\)\.$($_)_final_$version\b").Count | Should -Be 1 -Because "$_ data ingested under the $version schema must surface exactly once in the v1_4 hub function" + } + } + + It 'Reads <_> (new in FOCUS 1.4) from the v1_4 final table only' -ForEach $v14OnlyDatasets { + [regex]::Matches($hubFiles.v1_4, "database\('Ingestion'\)\.$($_)_final_v1_4\b").Count | Should -Be 1 + $hubFiles.v1_4 | Should -Not -Match "$($_)_final_v1_0" + $hubFiles.v1_4 | Should -Not -Match "$($_)_final_v1_2" + } + } + + Context 'HubSetup_v1_0.kql / HubSetup_v1_2.kql back-compat arms' { + + It 'HubSetup_v1_0.kql unions <_>_final_v1_4 (down-converted for old reports)' -ForEach $managedDatasets { + [regex]::Matches($hubFiles.v1_0, "database\('Ingestion'\)\.$($_)_final_v1_4\b").Count | Should -Be 1 -Because "data ingested under the v1_4 schema must remain visible to consumers still on the v1_0 functions" + } + + It 'HubSetup_v1_2.kql unions <_>_final_v1_4 (down-converted for old reports)' -ForEach $managedDatasets { + [regex]::Matches($hubFiles.v1_2, "database\('Ingestion'\)\.$($_)_final_v1_4\b").Count | Should -Be 1 -Because "data ingested under the v1_4 schema must remain visible to consumers still on the v1_2 functions" + } + } + + Context 'Provider column rename round trip (D3)' { + + # FOCUS 1.3 renamed ProviderName -> HostProviderName and repurposed PublisherName -> + # ServiceProviderName. Down-conversion (v1_4 data through v1_0/v1_2 functions) must map back: + # ProviderName = HostProviderName, PublisherName = ServiceProviderName. + It 'HubSetup_.kql down-converts the provider columns in its v1_4 arm' -ForEach @( + @{ Key = 'v1_0' } + @{ Key = 'v1_2' } ) { - $hubFiles.v1_4 | Should -Match "database\('Ingestion'\)\.$_" + $hubFiles[$Key] | Should -Match 'ProviderName\s*=\s*HostProviderName' -Because 'the deprecated ProviderName maps back from HostProviderName' + $hubFiles[$Key] | Should -Match 'PublisherName\s*=\s*ServiceProviderName' -Because 'the deprecated PublisherName maps back from ServiceProviderName' + } + + # The inverse guard is scoped to DIRECT single assignments (HostProviderName = PublisherName / + # ServiceProviderName = ProviderName) because the up-conversion in IngestionSetup_v1_4.kql and + # HubSetup_v1_4.kql legitimately uses the deprecated ProviderName as a *later fallback* inside + # case()/iff() expressions when deriving ServiceProviderName. A direct assignment is only ever + # the swapped-mapping defect this test exists to catch (PR #2126 review finding C1). + It ' does not invert the provider column mapping' -ForEach $v14SetupFiles { + $content = Get-Content -Path $FullName -Raw + $content | Should -Not -Match '(? to v1_4 (latest GA)' -ForEach @( - 'BillingPeriods', 'CommitmentDiscountUsage', 'ContractCommitments', - 'Costs', 'InvoiceDetails', 'Prices', 'Recommendations', 'Transactions' - ) { - $hubFiles.Latest | Should -Match "(?ms)$_\(\)\s*\{\s*${_}_v1_4\(\)\s*\}" + It 'Aliases <_>() to <_>_v1_4() (latest GA)' -ForEach $allDatasets { + $hubFiles.Latest | Should -Match "(?ms)$($_)\(\)\s*\{\s*$($_)_v1_4\(\)\s*\}" } It 'Does NOT alias to singular ContractCommitment' { @@ -246,21 +493,11 @@ Describe 'HubsFocusSchemas' { It 'Does NOT alias to v1_2 or older' { $hubFiles.Latest | Should -Not -Match '_v1_2\(\)' - } - } - - Context 'Bicep wiring' { - - It 'app.bicep loads <_>' -ForEach @( - 'IngestionSetup_v1_4.kql', 'HubSetup_v1_4.kql' - ) { - $appBicep | Should -Match ([regex]::Escape($_)) + $hubFiles.Latest | Should -Not -Match '_v1_0\(\)' } - It '.build.config lists <_>' -ForEach @( - 'IngestionSetup_v1_4.kql', 'HubSetup_v1_4.kql' - ) { - $buildConfig | Should -Match ([regex]::Escape($_)) + It 'Points all 8 unversioned functions at _v1_4()' { + [regex]::Matches($hubFiles.Latest, '_v1_4\(\)').Count | Should -Be 8 -Because 'every unversioned dataset function aliases exactly one _v1_4 function' } } } From 309ceb3f46050137228bbc151fda6098e6ac7de1 Mon Sep 17 00:00:00 2001 From: Michael Flanakin Date: Wed, 15 Jul 2026 17:10:30 -0700 Subject: [PATCH 7/8] test: update ContractCommitments golden lists to FOCUS 1.4 spec IDs Updates HubsFocusSchemas.Tests.ps1 to pin the spec-prefixed column names (30 FOCUS columns, including new ContractCommitmentDescription and ServiceProviderName), asserts the old unprefixed names are gone, and adds exact 30-column count checks for the raw and final tables. Also updates the changelog dataset column count (28 -> 30) and notes the spec-aligned naming. Co-Authored-By: Claude Fable 5 --- docs-mslearn/toolkit/changelog.md | 2 +- .../Tests/Unit/HubsFocusSchemas.Tests.ps1 | 48 ++++++++++++++----- 2 files changed, 37 insertions(+), 13 deletions(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 2c46c513e..6484e4a02 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -60,7 +60,7 @@ _Released June 2026_ - New versioned Hub database functions, like `Costs_v1_4()`, were added based on the FOCUS 1.4 schema changes. - Make sure you use versioned functions, like `Costs_v1_2()`, to avoid breaking changes between FOCUS versions. - Added new columns to the Costs managed dataset: `AllocatedMethodId`, `AllocatedMethodDetails`, `AllocatedResourceId`, `AllocatedResourceName`, `AllocatedTags`, `ContractApplied`, `ServiceProviderName`, `HostProviderName`, `CommitmentProgramEligibilityDetails`, `InvoiceDetailId`, and 12 `ContractCommitment*` per-row columns (`ContractCommitmentBenefitCategory`, `ContractCommitmentCreated`, `ContractCommitmentDiscountPercentage`, `ContractCommitmentDurationType`, `ContractCommitmentFulfillmentInterval`, `ContractCommitmentLastUpdated`, `ContractCommitmentLifecycleStatus`, `ContractCommitmentModel`, `ContractCommitmentOfferCategory`, `ContractCommitmentPaymentInterval`, `ContractCommitmentPaymentModel`, `ContractCommitmentPaymentUpfrontPercentage`). - - Added three new FOCUS 1.4 datasets: `ContractCommitments` (28 columns), `BillingPeriods` (6 columns), and `InvoiceDetails` (22 columns), each with tables and versioned and unversioned functions (for example, `ContractCommitments()` and `ContractCommitments_v1_4()`). + - Added three new FOCUS 1.4 datasets: `ContractCommitments` (30 columns, using FOCUS 1.4 specification column IDs like `ContractCommitmentBenefitCategory` and `ContractCommitmentLifecycleStatus`), `BillingPeriods` (6 columns), and `InvoiceDetails` (22 columns), each with tables and versioned and unversioned functions (for example, `ContractCommitments()` and `ContractCommitments_v1_4()`). - These datasets remain empty until Cost Management supports exporting FOCUS 1.4 data. - Added a static test harness for the hub database setup scripts that verifies build registration, schema version consistency, and update policy state to catch regressions before release. - **Changed** diff --git a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 index 599c2c662..aed18feac 100644 --- a/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 +++ b/src/powershell/Tests/Unit/HubsFocusSchemas.Tests.ps1 @@ -306,22 +306,38 @@ Describe 'HubsFocusSchemas' { It 'Includes base column <_>' -ForEach @( 'BillingCurrency', 'ContractCommitmentCategory', 'ContractCommitmentCost', - 'ContractCommitmentId', 'ContractCommitmentPeriodEnd', 'ContractCommitmentPeriodStart', - 'ContractCommitmentQuantity', 'ContractCommitmentType', 'ContractCommitmentUnit', - 'ContractId', 'ContractPeriodEnd', 'ContractPeriodStart', 'InvoiceIssuerName', - 'PricingCurrency' + 'ContractCommitmentDescription', 'ContractCommitmentId', 'ContractCommitmentPeriodEnd', + 'ContractCommitmentPeriodStart', 'ContractCommitmentQuantity', 'ContractCommitmentType', + 'ContractCommitmentUnit', 'ContractId', 'ContractPeriodEnd', 'ContractPeriodStart', + 'InvoiceIssuerName', 'PricingCurrency' ) { $contractCommitmentsRawBlock | Should -Match "(?m)^\s+$_\s*:" } It 'Includes FOCUS 1.4 column <_>' -ForEach @( - 'BenefitCategory', 'ContractCommitmentApplicability', 'Created', - 'DiscountPercentage', 'DurationType', 'FulfillmentInterval', 'LastUpdated', - 'LifecycleStatus', 'Model', 'OfferCategory', 'PaymentInterval', 'PaymentModel', - 'PaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost' + 'ContractCommitmentApplicability', 'ContractCommitmentBenefitCategory', + 'ContractCommitmentCreated', 'ContractCommitmentDiscountPercentage', + 'ContractCommitmentDurationType', 'ContractCommitmentFulfillmentInterval', + 'ContractCommitmentLastUpdated', 'ContractCommitmentLifecycleStatus', + 'ContractCommitmentModel', 'ContractCommitmentOfferCategory', + 'ContractCommitmentPaymentInterval', 'ContractCommitmentPaymentModel', + 'ContractCommitmentPaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost', + 'ServiceProviderName' ) { $contractCommitmentsRawBlock | Should -Match "(?m)^\s+$_\s*:" } + + It 'Does NOT include unprefixed column <_>' -ForEach @( + 'BenefitCategory', 'Created', 'DiscountPercentage', 'DurationType', + 'FulfillmentInterval', 'LastUpdated', 'LifecycleStatus', 'Model', 'OfferCategory', + 'PaymentInterval', 'PaymentModel', 'PaymentUpfrontPercentage' + ) { + $contractCommitmentsRawBlock | Should -Not -Match "(?m)^\s+$_\s*:" + } + + It 'Defines exactly 30 FOCUS columns' { + ([regex]::Matches($contractCommitmentsRawBlock, '(?m)^\s+(?!x_)\w+\s*:')).Count | Should -Be 30 + } } Context 'BillingPeriods_raw exists with FOCUS 1.4 columns' { @@ -412,13 +428,21 @@ Describe 'HubsFocusSchemas' { } It 'ContractCommitments_final_v1_4 includes FOCUS 1.4 column <_>' -ForEach @( - 'BenefitCategory', 'ContractCommitmentApplicability', 'Created', - 'DiscountPercentage', 'DurationType', 'FulfillmentInterval', 'LastUpdated', - 'LifecycleStatus', 'Model', 'OfferCategory', 'PaymentInterval', 'PaymentModel', - 'PaymentUpfrontPercentage', 'PricingCurrencyContractCommitmentCost' + 'ContractCommitmentApplicability', 'ContractCommitmentBenefitCategory', + 'ContractCommitmentCreated', 'ContractCommitmentDescription', + 'ContractCommitmentDiscountPercentage', 'ContractCommitmentDurationType', + 'ContractCommitmentFulfillmentInterval', 'ContractCommitmentLastUpdated', + 'ContractCommitmentLifecycleStatus', 'ContractCommitmentModel', + 'ContractCommitmentOfferCategory', 'ContractCommitmentPaymentInterval', + 'ContractCommitmentPaymentModel', 'ContractCommitmentPaymentUpfrontPercentage', + 'PricingCurrencyContractCommitmentCost', 'ServiceProviderName' ) { $contractCommitmentsFinalV14Block | Should -Match "(?m)^\s+$_\s*:" } + + It 'ContractCommitments_final_v1_4 defines exactly 30 FOCUS columns' { + ([regex]::Matches($contractCommitmentsFinalV14Block, '(?m)^\s+(?!x_)\w+\s*:')).Count | Should -Be 30 + } } Context 'HubSetup_v1_4.kql' { From 9657618e9f2fac9e715ed8f66b9f95c85e06d4ff Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 16 Jul 2026 00:11:21 +0000 Subject: [PATCH 8/8] chore: Update ms.date in docs-mslearn files --- docs-mslearn/toolkit/changelog.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs-mslearn/toolkit/changelog.md b/docs-mslearn/toolkit/changelog.md index 6484e4a02..e168935bc 100644 --- a/docs-mslearn/toolkit/changelog.md +++ b/docs-mslearn/toolkit/changelog.md @@ -3,7 +3,7 @@ title: FinOps toolkit changelog description: Review the latest features and enhancements in the FinOps toolkit, including updates to FinOps hubs, Power BI reports, and more. author: MSBrett ms.author: brettwil -ms.date: 07/15/2026 +ms.date: 07/16/2026 ms.topic: reference ms.service: finops ms.subservice: finops-toolkit