Add FOCUS 1.4 hub functions (HubSetup_v1_4.kql) by flanakin · Pull Request #2128 · microsoft/finops-toolkit

flanakin · 2026-05-06T15:22:04Z

🛠️ Description

Adds FOCUS 1.4 hub database functions with full cross-version compatibility. Part 2 of the FOCUS 1.4 PR stack; based on #2126 (phase 1, ingestion), so all IngestionSetup_* content in this diff comes from that PR — review only HubSetup_*, dashboard.json, and the hub-side app.bicep/.build.config entries here.

New HubSetup_v1_4.kql with {ds}_v1_4() functions for all 8 datasets: BillingPeriods, CommitmentDiscountUsage, ContractCommitments, Costs, InvoiceDetails, Prices, Recommendations, Transactions.
- Each managed dataset unions _final_v1_4 ∪ _final_v1_2 ∪ _final_v1_0, up-converting older rows (decimal→real, column renames, FOCUS 1.4 column defaults).
- Costs_v1_4() derives HostProviderName/ServiceProviderName from deprecated ProviderName/PublisherName, mirroring the ingestion mapping (non-empty guaranteed per FOCUS).
- The 3 new FOCUS 1.4 datasets (BillingPeriods, ContractCommitments, InvoiceDetails) read _final_v1_4 only; they remain empty until Cost Management ships an export.
HubSetup_v1_0.kql / HubSetup_v1_2.kql: all 5 existing dataset functions now also union _final_v1_4 down-converted (ProviderName = HostProviderName, PublisherName = ServiceProviderName, FOCUS 1.3/1.4-only columns removed, real→decimal for v1_0), so existing Power BI reports and dashboards pinned to _v1_0()/_v1_2() keep working after ingestion switches to 1.4.
HubSetup_Latest.kql aliases all 8 unversioned functions (Costs(), Prices(), etc.) to _v1_4(); unversioned tails removed from versioned files (Latest is the single owner).
dashboard.json repointed from Costs_v1_2 to Costs_v1_4.
CommitmentDiscountUsage_v1_4() Services lookup uses take_any to prevent row fan-out (same class as Fix Costs_v1_2 duplicate records from Services table join #1907).
app.bicep and .build.config register the v1_4 hub script.

Verification

pwsh -Command ./src/scripts/Build-Toolkit finops-hub passes (Bicep compiles; KQL under loadTextContent() size limits).
Each of the 5 managed {ds}_v1_4() functions references _final_v1_0, _final_v1_2, and _final_v1_4.
Each of the 5 dataset functions in HubSetup_v1_0.kql and HubSetup_v1_2.kql references _final_v1_4.
HubSetup_Latest.kql aliases all 8 unversioned functions to _v1_4(); no unversioned definitions remain in HubSetup_v1_2.kql/HubSetup_v1_4.kql.
dashboard.json has zero Costs_v1_2 references.
Verified _final_v1_4 table schemas match _final_v1_2 for CommitmentDiscountUsage/Prices/Recommendations/Transactions (plain unions safe); Costs delta (23 new columns, 2 removed) handled explicitly.
Manual ADX deployment validation deferred to the umbrella PR gate per plan D6.

📋 Checklist

🔬 How did you test this change?

🤏 Lint tests

🤞 PS -WhatIf / az validate

👍 Manually deployed + verified

💪 Unit tests

🙌 Integration tests

📦 Deploy to test?

Hubs + ADX (managed)

Hubs + Fabric (manual) — URI:

Hubs (manual)

Hubs (no data)

Workbooks

Alerts

🙋‍♀️ Do any of the following that apply?

🚨 This is a breaking change.

🤏 The change is less than 20 lines of code.

🤖 Generated with Claude Code

Adds HubSetup_v1_3.kql with Costs_v1_3, Prices_v1_3, CommitmentDiscountUsage_v1_3, Recommendations_v1_3, and Transactions_v1_3 that union the new Costs_final_v1_3 with the existing Costs_final_v1_2 and Costs_final_v1_0 tables. For v1_2 data unioned into the v1_3 view, the 8 new FOCUS 1.3 columns default to null/empty and ServiceProviderName/HostProviderName are populated from the deprecated ProviderName/PublisherName for back compat. The same defaults apply to the v1_0 union arm on top of the existing v1_0 -> v1_2 conversion. HubSetup_Latest.kql aliases now point to *_v1_3 functions so the unversioned Costs(), Prices(), etc. return the latest schema. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…cus14-phase2-hubs-1.3

Renames HubSetup_v1_3.kql to HubSetup_v1_4.kql, consolidating all hub functions (including ContractCommitment) for FOCUS 1.4 GA. Updates HubSetup_Latest.kql to alias all unversioned functions to v1_4. Updates app.bicep and .build.config to reference v1_4. Co-Authored-By: Claude <noreply@anthropic.com>

…cus14-phase2-hubs-1.3

- Rename ContractCommitment_v1_4() -> ContractCommitments_v1_4() (and update folder label) - Add BillingPeriods_v1_4() and InvoiceDetails_v1_4() hub functions - Add BillingPeriods(), ContractCommitments() (renamed), and InvoiceDetails() unversioned aliases in HubSetup_Latest.kql - Add CommitmentProgramEligibilityDetails + 12 ContractCommitment* + InvoiceDetailId column defaults to Costs_final_v1_0 and Costs_final_v1_2 union arms (back-compat shims) and add them to the project list 🤖 Generated with [Claude Code](https://claude.ai/claude-code) Co-Authored-By: Michael Flanakin <flanakin@users.noreply.github.com> Co-Authored-By: Claude <noreply@anthropic.com>

flanakin · 2026-05-26T07:25:05Z

🤖 [AI][Claude Code] PR Update Summary

Cascade of PR #2126 feedback (#2126 review threads 1, 4, 5, 6):

✅ Pluralized ContractCommitment_v1_4() -> ContractCommitments_v1_4()
✅ Added BillingPeriods_v1_4() and InvoiceDetails_v1_4() hub functions with matching HubSetup_Latest aliases
✅ Added new FOCUS 1.4 column defaults (CommitmentProgramEligibilityDetails, 12 ContractCommitment* columns, InvoiceDetailId) to the Costs_v1_4() back-compat union arms
✅ Updated Costs_v1_4() project list to include the new columns

Depends on PR #2126 (ingestion side).

flanakin · 2026-05-27T06:46:27Z

Should this be in this PR? It's already in PR #2127

flanakin · 2026-05-27T06:47:15Z

Same comment: This is already in PR @2126

Do we want to review it here or there?

flanakin · 2026-05-27T06:53:35Z

+// BillingPeriods_final_v1_4
+.create-or-alter function
+with (docstring = 'Gets all billing periods aligned to FOCUS 1.4.', folder = 'BillingPeriods')
+BillingPeriods_v1_4()


Sort this list alphabetically. BillingPeriods should be before CommitmentDiscountUsage and InvoiceDetails should be after Costs.

flanakin · 2026-05-27T06:54:42Z

We're missing the updates to the HubSetup_v1_0.kql and HubSetup_v1_2.kql functions to backport v1_4 data to those schemas.

flanakin · 2026-05-28T10:08:51Z

+        PricingCurrency,
+        PricingQuantity,
+        PricingUnit,
+        // ProviderName and PublisherName removed in FOCUS 1.4. Use ServiceProviderName / HostProviderName.


Suggested change

// ProviderName and PublisherName removed in FOCUS 1.4. Use ServiceProviderName / HostProviderName.

flanakin · 2026-05-28T10:11:09Z

+
+
+//======================================================================================================================
+// Latest FOCUS version


This section doesn't belong in this file. It's in the HubSetup_Latest.kql file already

flanakin · 2026-05-28T10:25:22Z

+with (docstring = 'Gets all commitment discount usage records aligned to FOCUS 1.4.', folder = 'CommitmentDiscountUsage')
+CommitmentDiscountUsage_v1_4()
+{
+    database('Ingestion').CommitmentDiscountUsage_final_v1_4


Suggested change

database('Ingestion').CommitmentDiscountUsage_final_v1_4

database('Ingestion').CommitmentDiscountUsage_final_v1_4

| union (database('Ingestion').CommitmentDiscountUsage_final_v1_2)

flanakin · 2026-05-28T10:42:20Z

+with (docstring = 'Gets all prices aligned to FOCUS 1.4.', folder = 'Prices')
+Prices_v1_4()
+{
+    database('Ingestion').Prices_final_v1_4


Suggested change

database('Ingestion').Prices_final_v1_4

database('Ingestion').Prices_final_v1_4

| union (database('Ingestion').Prices_final_v1_2)

flanakin · 2026-05-28T10:42:56Z

+with (docstring = 'Gets all recommendations aligned to FOCUS 1.4.', folder = 'Recommendations')
+Recommendations_v1_4()
+{
+    database('Ingestion').Recommendations_final_v1_4


Suggested change

database('Ingestion').Recommendations_final_v1_4

database('Ingestion').Recommendations_final_v1_4

| union (database('Ingestion').Recommendations_final_v1_2)

flanakin · 2026-05-28T10:43:43Z

+with (docstring = 'Gets all transactions aligned to FOCUS 1.4.', folder = 'Transactions')
+Transactions_v1_4()
+{
+    database('Ingestion').Transactions_final_v1_4


Suggested change

database('Ingestion').Transactions_final_v1_4

database('Ingestion').Transactions_final_v1_4

| union (database('Ingestion').Transactions_final_v1_2)

…n-1.3' into flanakin/focus14-phase2-hubs-1.3

- Union _final_v1_2 tables into CommitmentDiscountUsage/Prices/Recommendations/Transactions_v1_4() to avoid data loss for existing hubs - Add _final_v1_4 down-convert arms to all 5 dataset functions in HubSetup_v1_0.kql and HubSetup_v1_2.kql for back compat with reports pinned to older schemas - Fix swapped HostProviderName/ServiceProviderName up-converts in Costs_v1_4() to mirror ingestion mapping - Remove duplicated unversioned function tails from HubSetup_v1_4.kql and HubSetup_v1_2.kql (HubSetup_Latest.kql is the single owner) - Repoint dashboard.json from Costs_v1_2 to Costs_v1_4 - Harden CommitmentDiscountUsage_v1_4() Services lookup with take_any to prevent row fan-out - Alphabetize function order, remove stale comment, and fix trailing whitespace in HubSetup_v1_4.kql Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

flanakin · 2026-07-15T20:15:42Z

Pushed 7181c9a addressing all review feedback (after merging in the #2126 round-2 fixes, cd2873b). Mapping to review comments:

Comment	Fix
3308953372, 3308956972 (IngestionSetup content belongs to #2126/#2127)	PR base retargeted to `flanakin/focus14-phase1-ingestion-1.3`, so the diff now shows only hub-side changes; ingestion files are reviewed in #2126
3308989211 (alphabetize)	Functions in `HubSetup_v1_4.kql` reordered: BillingPeriods, CommitmentDiscountUsage, ContractCommitments, Costs, InvoiceDetails, Prices, Recommendations, Transactions
3308994843 (backport to v1_0/v1_2)	All 5 dataset functions in `HubSetup_v1_0.kql` and `HubSetup_v1_2.kql` now union `_final_v1_4` down-converted. Verified `_final_v1_4` schemas are identical to `_final_v1_2` for CommitmentDiscountUsage/Prices/Recommendations/Transactions, so those arms only need the existing v1_0 type/rename conversions; Costs restores `ProviderName = HostProviderName` / `PublisherName = ServiceProviderName` and removes the 23 FOCUS 1.3/1.4-only columns. The 3 new datasets (BillingPeriods/ContractCommitments/InvoiceDetails) are not backported since those schema versions never had them.
3317092338, 3317188755, 3317191440 + 3317005762, 3317195217 (missing `_final_v1_2` unions)	Suggestions applied as-is to CommitmentDiscountUsage/Prices/Recommendations/Transactions_v1_4() (`Costs_v1_4()` already had its v1_2 arm)
3316998842 (inline comment in project list)	Removed
3317011679 (unversioned tail)	Removed from `HubSetup_v1_4.kql`, and also removed the stale equivalent tail from `HubSetup_v1_2.kql` — `HubSetup_Latest.kql` is the single owner of the unversioned `{ds}()` aliases (all 8 verified pointing to `_v1_4()`)

Also in this push (same class as prior review findings):

Provider mapping fix: both up-convert arms in Costs_v1_4() had HostProviderName/ServiceProviderName swapped; now HostProviderName = ProviderName and ServiceProviderName = PublisherName → ProviderName → 'Microsoft', mirroring the ingestion mapping fixed in Add FOCUS 1.4 ingestion (IngestionSetup_v1_4.kql) #2126
Fan-out hardening: CommitmentDiscountUsage_v1_4() Services lookup switched from distinct to the take_any pattern used in Costs_v1_4() to prevent row duplication (Fix Costs_v1_2 duplicate records from Services table join #1907 class). Note: the same distinct lookup exists in the pre-existing CommitmentDiscountUsage_v1_2() (HubSetup_v1_2.kql L27); left untouched here to avoid changing shipped v1_2 behavior in this PR — happy to include if preferred
dashboard.json repointed Costs_v1_2 → Costs_v1_4 (2 references)

Verification: Build-Toolkit finops-hub exit 0; zero conflict markers; zero v1_3 references in hub scripts; all HubSetup files well under the 131 K loadTextContent() limit (largest 27.5 K). Changelog entries deferred to #2136 per the plan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

flanakin · 2026-07-16T00:05:45Z

Synced with phase 1 (which now carries the dev merge, the v1_2→v1_4 fix ports, and the Q4 ContractCommitments spec alignment). Clean merge, no conflicts.

No hub-function changes were needed: ContractCommitments_v1_4() in HubSetup_v1_4.kql is a pass-through of ContractCommitments_final_v1_4 with no explicit column projection, so it automatically surfaces the renamed/added spec columns. The flattened ContractCommitment* per-row columns in the Costs functions are untouched (separate design).

Verified: Build-Toolkit finops-hub exit 0; no conflict markers; v1_4 KQL sizes within limits.

Copilot AI review requested due to automatic review settings May 6, 2026 15:22

flanakin requested review from MSBrett and RolandKrummenacher as code owners May 6, 2026 15:22

microsoft-github-policy-service Bot requested a review from helderpinto May 6, 2026 15:22

microsoft-github-policy-service Bot added the Needs: Review 👀 PR that is ready to be reviewed label May 6, 2026

Copilot started reviewing on behalf of flanakin May 6, 2026 15:22 View session

microsoft-github-policy-service Bot assigned MSBrett May 6, 2026

microsoft-github-policy-service Bot added the Tool: FinOps hubs Data pipeline solution label May 6, 2026

microsoft-github-policy-service Bot assigned helderpinto and RolandKrummenacher May 6, 2026

This was referenced May 6, 2026

Add ContractCommitment dataset #2130

Closed

FOCUS 1.3 and 1.4 support across the toolkit #2120

Open

flanakin changed the base branch from flanakin/focus14-phase1-ingestion-1.3 to flanakin/focus14 May 6, 2026 23:00

flanakin marked this pull request as draft May 6, 2026 23:16

flanakin added this to the v15 milestone May 15, 2026

flanakin and others added 2 commits May 17, 2026 13:24

Merge branch 'flanakin/focus14-phase1-ingestion-1.3' into flanakin/fo…

7d9d295

…cus14-phase2-hubs-1.3

flanakin changed the title ~~Add FOCUS 1.3 hub functions (HubSetup_v1_3.kql)~~ Add FOCUS 1.4 hub functions (HubSetup_v1_4.kql) May 17, 2026

flanakin mentioned this pull request May 17, 2026

Add FOCUS 1.4-preview hub functions (HubSetup_v1_4.kql) #2134

Closed

3 tasks

flanakin mentioned this pull request May 26, 2026

Add FOCUS 1.4 ingestion (IngestionSetup_v1_4.kql) #2126

Draft

flanakin and others added 2 commits May 26, 2026 00:22

Merge branch 'flanakin/focus14-phase1-ingestion-1.3' into flanakin/fo…

6f2ec90

…cus14-phase2-hubs-1.3

This was referenced May 26, 2026

Refresh Claude Code plugin skill files for FOCUS 1.4 #2135

Draft

Add FOCUS 1.4 schema tests and changelog entries #2136

Draft

flanakin commented May 28, 2026

View reviewed changes

This was referenced Jul 15, 2026

Add FOCUS 1.4 support #2207

Draft

Refresh FinOps hub upgrade procedure for multi-version FOCUS adoption #2123

Open

Merge remote-tracking branch 'origin/flanakin/focus14-phase1-ingestio…

cd2873b

…n-1.3' into flanakin/focus14-phase2-hubs-1.3

flanakin changed the base branch from flanakin/focus14 to flanakin/focus14-phase1-ingestion-1.3 July 15, 2026 20:14

flanakin mentioned this pull request Jul 15, 2026

Update Power BI reports to FOCUS 1.4 (v1_4 schema) #2209

Draft

merge: sync flanakin/focus14-phase1-ingestion-1.3 into phase2

177a640

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>



		//======================================================================================================================
		// Latest FOCUS version

	database('Ingestion').CommitmentDiscountUsage_final_v1_4
	database('Ingestion').CommitmentDiscountUsage_final_v1_4
	\| union (database('Ingestion').CommitmentDiscountUsage_final_v1_2)

	database('Ingestion').Prices_final_v1_4
	database('Ingestion').Prices_final_v1_4
	\| union (database('Ingestion').Prices_final_v1_2)

	database('Ingestion').Recommendations_final_v1_4
	database('Ingestion').Recommendations_final_v1_4
	\| union (database('Ingestion').Recommendations_final_v1_2)

	database('Ingestion').Transactions_final_v1_4
	database('Ingestion').Transactions_final_v1_4
	\| union (database('Ingestion').Transactions_final_v1_2)

Uh oh!

Conversation

flanakin commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🛠️ Description

Verification

📋 Checklist

🔬 How did you test this change?

📦 Deploy to test?

🙋‍♀️ Do any of the following that apply?

Uh oh!

flanakin commented May 26, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

flanakin commented Jul 15, 2026

Uh oh!

flanakin commented Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

flanakin commented May 6, 2026 •

edited

Loading