Skip to content

Add FOCUS 1.4 ingestion (IngestionSetup_v1_4.kql)#2126

Draft
flanakin wants to merge 7 commits into
flanakin/focus14from
flanakin/focus14-phase1-ingestion-1.3
Draft

Add FOCUS 1.4 ingestion (IngestionSetup_v1_4.kql)#2126
flanakin wants to merge 7 commits into
flanakin/focus14from
flanakin/focus14-phase1-ingestion-1.3

Conversation

@flanakin

@flanakin flanakin commented May 6, 2026

Copy link
Copy Markdown
Collaborator

🛠️ Description

Adds the FOCUS 1.4 ingestion schema (v1_4) for FinOps hubs, converting all currently supported export versions up to FOCUS 1.4 and pre-wiring detection for native 1.3/1.4 exports.

  • Adds IngestionSetup_v1_4.kql (Prices, Costs, ActualCosts/AmortizedCosts C360 transforms) and IngestionSetup_v1_4_Datasets.kql (BillingPeriods, CommitmentDiscountUsage, ContractCommitments, InvoiceDetails, Recommendations, Transactions) — split into two self-contained files to stay under the Bicep loadTextContent() 131,072-character limit; each dataset's transform, final table, and update policy live in the same file.
  • Adds FOCUS 1.3/1.4 superset columns to Costs_raw and ContractCommitments_raw in IngestionSetup_RawTables.kql.
  • Maps removed ProviderName/PublisherName to HostProviderName/ServiceProviderName (falls back to the deprecated columns; guarantees non-null ServiceProviderName per spec); original values are always preserved in x_SourceValues.
  • Detects native FOCUS 1.3/1.4 exports in x_SourceVersion (by ServiceProviderName/HostProviderName/InvoiceDetailId presence) and treats 1.4 as the current version when labeling x_SourceChanges.
  • Projects all 22 new Costs_raw columns from the C360 (ActualCosts/AmortizedCosts) transforms in v1_0, v1_2, and v1_4 so the transactional update policies validate against the raw table schema.
  • Disables all v1_2 update policies and marks the v1_2 transform docstrings deprecated, pointing to the _v1_4() transforms (same pattern as the v1_0 → v1_2 upgrade).
  • Registers both v1_4 scripts in Analytics/app.bicep and the .build.config Fabric KQL bundle.

New FOCUS 1.4 datasets (BillingPeriods, ContractCommitments, InvoiceDetails) define schema in advance; they remain empty until Cost Management ships an export for them.

Verification

  • Build-Toolkit finops-hub succeeds (bicep build + loadTextContent; both v1_4 files under the 131,072-character limit: 91,437 and 37,175 bytes).
  • Both C360 transforms in v1_0, v1_2, and v1_4 project the exact Costs_raw column set (166 columns; verified programmatically).
  • IngestionSetup_v1_2.kql has zero enabled update policies; v1_4 files have 4 + 6 enabled policies.
  • Fabric KQL bundle (finops-hub-fabric-setup-Ingestion.kql) includes both v1_4 files.
  • HubsIngestionQueries.Tests.ps1: 192 passed, 0 failed.
  • Manual ADX deployment validation (all supported export versions) is tracked for the umbrella PR.

📋 Checklist

🔬 How did you test this change?

  • 🤏 Lint tests
  • 🤞 PS -WhatIf / az validate
  • 👍 Manually deployed + verified
  • 💪 Unit tests
  • 🙌 Integration tests

📦 Deploy to test?

  • Hubs + ADX (managed)
  • Hubs + Fabric (manual) — URI:
  • Hubs (manual)
  • Hubs (no data)
  • Workbooks
  • Alerts

🙋‍♀️ Do any of the following that apply?

  • 🚨 This is a breaking change.
  • 🤏 The change is less than 20 lines of code.

🤖 Generated with Claude Code

Adds IngestionSetup_v1_3.kql with Costs/Prices/CommitmentDiscountUsage/
Recommendations/Transactions transforms and final tables renamed for
FOCUS 1.3. Cost and Usage gains 8 new FOCUS 1.3 columns:
AllocatedMethodId/Details/ResourceId/ResourceName/Tags (data-generator
split cost allocation), ContractApplied (per-row contract commitment
application), ServiceProviderName + HostProviderName (replacing the
deprecated ProviderName/PublisherName, with empty-fallback for back
compat).

Costs_raw now carries the new columns so downstream v1_2 transforms
keep working and v1_3 transforms can read them directly. The v1_3
file is wired into the Bicep deployment alongside v1_0 and v1_2.

The unversioned Costs() function still aliases to v1_2 — phase 2
adds Costs_v1_3() and retargets Latest.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidates FOCUS 1.3 and 1.4 column additions into a single v1_4 ingestion
setup file. Removes the intermediate v1_3 file, adds ContractCommitment_raw
table definition, and updates app.bicep and .build.config to reference v1_4.

Co-Authored-By: Claude <noreply@anthropic.com>
@flanakin flanakin changed the title Add FOCUS 1.3 ingestion (IngestionSetup_v1_3.kql) Add FOCUS 1.4 ingestion (IngestionSetup_v1_4.kql) May 17, 2026

@flanakin flanakin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes here need to also be applied to the other 3 FOCUS 1.4 PRs

@@ -359,6 +455,11 @@

// Costs_raw table -- Redefine all columns
.alter table Costs_raw (

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FOCUS 1.4 has a requirement to include ALL native columns. This will significantly bloat the table size for cross-cloud data. Given we don't have a full set of columns for this yet, we need to have a way to ingest any additional columns either by auto-accepting new columns on ingestion or dumping them into a single x_ column. We need to research options and weigh pros/cons. This can happen in a separate PR, but don't resolve this without a link to that PR.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 [AI][Claude Code] 🤔 Needs discussion

Filed a tracking issue to scope the research: #2159 — Research: Handle FOCUS 1.4 native-columns requirement to avoid raw table bloat. The issue captures the three candidate approaches (auto-accept new columns, single x_NativeColumns JSON, hybrid) and explicitly scopes implementation to a separate PR.

This PR ships the FOCUS 1.4 schema with the columns we know today; native-columns handling will follow #2159's resolution. Leaving this thread unresolved per your request until the follow-up PR is linked.

- Pluralize ContractCommitments_raw / _transform_v1_4 / _final_v1_4 / mapping / update policy
- Add BillingPeriods_raw + IngestionSetup_v1_4 transforms/finals/policies (FOCUS 1.4 supplemental dataset)
- Add InvoiceDetails_raw + IngestionSetup_v1_4 transforms/finals/policies (FOCUS 1.4 supplemental dataset)
- Add 12 ContractCommitment* per-row columns + CommitmentProgramEligibilityDetails + InvoiceDetailId to Costs_raw, Costs_final_v1_4, and Costs_transform_v1_4
- Fix x_Source* annotations on ContractCommitments_raw to Hubs v1_4+
- Update Costs_raw ProviderName/PublisherName annotations to FOCUS 0.5-1.3 (removed in 1.4; kept for back compat)
- Update Costs_raw Region annotation: deprecated -> removed
- File #2159 for the FOCUS 1.4 native-columns architecture research

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-Authored-By: Michael Flanakin <flanakin@users.noreply.github.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@flanakin

Copy link
Copy Markdown
Collaborator Author

🤖 [AI][Claude Code] PR Update Summary

Addressed: 7 thread(s)

Key changes:

  • Pluralized ContractCommitment_* -> ContractCommitments_* across tables, functions, mapping, and update policies (Thread 1)
  • Updated x_Source* annotations Hubs v1_3+ -> v1_4+ on ContractCommitments_raw (Thread 2)
  • Updated ProviderName/PublisherName/Region annotations to mark them as removed (Thread 3)
  • Added BillingPeriods_raw and InvoiceDetails_raw datasets with transforms/finals/policies (Thread 4)
  • Added 12 ContractCommitment* per-row columns + CommitmentProgramEligibilityDetails to Costs_raw (Thread 5)
  • Added InvoiceDetailId to Costs_raw (Thread 6)
  • Filed Research: Handle FOCUS 1.4 native-columns requirement to avoid raw table bloat #2159 for native-columns research (Thread 7)

Cascading next: Same pluralization + new BillingPeriods/InvoiceDetails datasets will be added to PRs #2128 (hub functions), #2135 (plugin docs), and #2136 (tests + changelog).

flanakin added a commit that referenced this pull request May 26, 2026
- Rename ContractCommitment_v1_4() -> ContractCommitments_v1_4() (and update folder label)
- Add BillingPeriods_v1_4() and InvoiceDetails_v1_4() hub functions
- Add BillingPeriods(), ContractCommitments() (renamed), and InvoiceDetails() unversioned aliases in HubSetup_Latest.kql
- Add CommitmentProgramEligibilityDetails + 12 ContractCommitment* + InvoiceDetailId column defaults to Costs_final_v1_0 and Costs_final_v1_2 union arms (back-compat shims) and add them to the project list

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-Authored-By: Michael Flanakin <flanakin@users.noreply.github.com>
Co-Authored-By: Claude <noreply@anthropic.com>
flanakin added a commit that referenced this pull request May 26, 2026
- Rename ContractCommitment() -> ContractCommitments() in skill references and agent docs
- Add BillingPeriods() and InvoiceDetails() to the function list and dedicated agent doc sections
- Mention the new FOCUS 1.4 per-row columns (CommitmentProgramEligibilityDetails, 12 ContractCommitment* columns, InvoiceDetailId)
- Update output-style guidance to reference all three supplemental datasets

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-Authored-By: Michael Flanakin <flanakin@users.noreply.github.com>
Co-Authored-By: Claude <noreply@anthropic.com>
flanakin added a commit that referenced this pull request May 27, 2026
Tests:
- Pluralize ContractCommitments_raw / _final_v1_4 / _v1_4 assertions
- Assert new BillingPeriods_raw + InvoiceDetails_raw datasets exist with their FOCUS 1.4 columns
- Assert new BillingPeriods_v1_4(), InvoiceDetails_v1_4(), ContractCommitments_v1_4() hub functions
- Assert HubSetup_Latest aliases for BillingPeriods(), ContractCommitments(), InvoiceDetails()
- Assert Costs_raw and Costs_final_v1_4 include the 14 new FOCUS 1.4 columns
- Assert NO singular ContractCommitment_raw / ContractCommitment_v1_4 / ContractCommitment() exists
- Assert Costs_raw keeps removed ProviderName/PublisherName/Region for back compat

Changelog:
- Expand v15.0.0 entry with the 14 new FOCUS 1.4 columns and the three new supplemental datasets (ContractCommitments, BillingPeriods, InvoiceDetails)
- Update plugin entry to list ContractCommitments() / BillingPeriods() / InvoiceDetails()

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-Authored-By: Michael Flanakin <flanakin@users.noreply.github.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Comment on lines +568 to +570
ProviderName: string, // FOCUS 0.5-1.3 (removed in 1.4; kept for back compat)
PublisherName: string, // FOCUS 0.5-1.3 (removed in 1.4; kept for back compat)
Region: string, // FOCUS 0.5-1.0-preview (removed; kept for back compat)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ProviderName: string, // FOCUS 0.5-1.3 (removed in 1.4; kept for back compat)
PublisherName: string, // FOCUS 0.5-1.3 (removed in 1.4; kept for back compat)
Region: string, // FOCUS 0.5-1.0-preview (removed; kept for back compat)
ProviderName: string, // FOCUS 0.5-1.3 (removed; keep for back-compat)
PublisherName: string, // FOCUS 0.5-1.3 (removed; keep for back-compat)
Region: string, // FOCUS 0.5-1.0-preview (removed; keep for back-compat)

Comment on lines +527 to +536
// FOCUS 1.3
isnotempty(CommitmentDiscountQuantity), CommitmentDiscountQuantity,
// FOCUS 1.0-preview, 1.0
isempty(CommitmentDiscountStatus), real(null),
CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),
CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,
real(null)
)
| extend old_CommitmentDiscountUnit = CommitmentDiscountUnit, CommitmentDiscountUnit = case(
// FOCUS 1.3

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// FOCUS 1.3
isnotempty(CommitmentDiscountQuantity), CommitmentDiscountQuantity,
// FOCUS 1.0-preview, 1.0
isempty(CommitmentDiscountStatus), real(null),
CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),
CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,
real(null)
)
| extend old_CommitmentDiscountUnit = CommitmentDiscountUnit, CommitmentDiscountUnit = case(
// FOCUS 1.3
// FOCUS 1.2+
isnotempty(CommitmentDiscountQuantity), CommitmentDiscountQuantity,
// FOCUS 1.0-preview, 1.0
isempty(CommitmentDiscountStatus), real(null),
CommitmentDiscountCategory == 'Spend', EffectiveCost / coalesce(x_BillingExchangeRate, real(1)),
CommitmentDiscountCategory == 'Usage' and isnotempty(x_CommitmentDiscountNormalizedRatio), PricingQuantity / coalesce(x_PricingBlockSize, real(1)) * x_CommitmentDiscountNormalizedRatio,
real(null)
)
| extend old_CommitmentDiscountUnit = CommitmentDiscountUnit, CommitmentDiscountUnit = case(
// FOCUS 1.2+

//
// Pricing
| extend old_x_AmortizationClass = x_AmortizationClass, x_AmortizationClass = case(
// FOCUS 1.3

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// FOCUS 1.3
// FOCUS 1.2+

)
| extend x_ConsumedCoreHours = iff(ConsumedUnit == 'Hours' and isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, real(null))
| extend SkuPriceDetails = case(
// FOCUS 1.3

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// FOCUS 1.3
// FOCUS 1.2+

```


//===| BillingPeriods |=================================================================================================

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BillingPeriods should be first in this file (alphabetized list)

Comment on lines +749 to +761
ContractApplied = parse_json(ContractApplied), // FOCUS 1.3+
ContractCommitmentBenefitCategory = column_ifexists('ContractCommitmentBenefitCategory', ''), // FOCUS 1.4+
ContractCommitmentCreated = column_ifexists('ContractCommitmentCreated', datetime(null)), // FOCUS 1.4+
ContractCommitmentDiscountPercentage = column_ifexists('ContractCommitmentDiscountPercentage', real(null)), // FOCUS 1.4+
ContractCommitmentDurationType = column_ifexists('ContractCommitmentDurationType', ''), // FOCUS 1.4+
ContractCommitmentFulfillmentInterval = column_ifexists('ContractCommitmentFulfillmentInterval', ''), // FOCUS 1.4+
ContractCommitmentLastUpdated = column_ifexists('ContractCommitmentLastUpdated', datetime(null)), // FOCUS 1.4+
ContractCommitmentLifecycleStatus = column_ifexists('ContractCommitmentLifecycleStatus', ''), // FOCUS 1.4+
ContractCommitmentModel = column_ifexists('ContractCommitmentModel', ''), // FOCUS 1.4+
ContractCommitmentOfferCategory = column_ifexists('ContractCommitmentOfferCategory', ''), // FOCUS 1.4+
ContractCommitmentPaymentInterval = column_ifexists('ContractCommitmentPaymentInterval', ''), // FOCUS 1.4+
ContractCommitmentPaymentModel = column_ifexists('ContractCommitmentPaymentModel', ''), // FOCUS 1.4+
ContractCommitmentPaymentUpfrontPercentage = column_ifexists('ContractCommitmentPaymentUpfrontPercentage', real(null)), // FOCUS 1.4+

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port from older FOCUS exports

ContractedUnitPrice = coalesce(ContractedUnitPrice, x_OnDemandUnitPrice),
EffectiveCost,
HostProviderName = iff(isempty(HostProviderName), PublisherName, HostProviderName), // FOCUS 1.3+; falls back to deprecated PublisherName for back compat
InvoiceDetailId = column_ifexists('InvoiceDetailId', ''), // FOCUS 1.4+

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port from older FOCUS exports

PricingCurrency = coalesce(PricingCurrency, x_PricingCurrency),
PricingQuantity,
PricingUnit,
// ProviderName and PublisherName removed in FOCUS 1.4. Use ServiceProviderName / HostProviderName.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// ProviderName and PublisherName removed in FOCUS 1.4. Use ServiceProviderName / HostProviderName.

ResourceType,
ServiceCategory,
ServiceName,
ServiceProviderName = iff(isempty(ServiceProviderName), ProviderName, ServiceProviderName), // FOCUS 1.3+; falls back to deprecated ProviderName for back compat

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ServiceProviderName = iff(isempty(ServiceProviderName), ProviderName, ServiceProviderName), // FOCUS 1.3+; falls back to deprecated ProviderName for back compat
ServiceProviderName = iff(isempty(ServiceProviderName), PublisherName, ServiceProviderName), // FOCUS 1.3+; falls back to deprecated PublisherName for back compat

PricingCurrency: string,
PricingQuantity: real,
PricingUnit: string,
// ProviderName and PublisherName removed in FOCUS 1.4

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// ProviderName and PublisherName removed in FOCUS 1.4

@flanakin flanakin removed the Needs: Review 👀 PR that is ready to be reviewed label May 28, 2026
@flanakin flanakin added the Needs: Attention 👋 Issue or PR needs to be reviewed by the author or it will be closed due to no activity label May 28, 2026
@flanakin flanakin mentioned this pull request Jul 15, 2026
30 tasks
flanakin added a commit that referenced this pull request Jul 15, 2026
- D4 revised: new FOCUS datasets in scope per owner review direction on #2126
- D3 refined: provider-name mapping non-null guarantee; swap defect noted
- D9 added: split IngestionSetup_v1_4.kql for the Bicep 131K loadTextContent limit
- §6: dispositions + blocking fix lists for all 6 reviewed PRs

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Fix swapped HostProviderName/ServiceProviderName fallbacks and guarantee non-null ServiceProviderName (C1)
- Project all 22 new Costs_raw columns from the ActualCosts/AmortizedCosts C360 transforms in v1_0, v1_2, and v1_4 (C2)
- Disable v1_2 update policies and add deprecation docstrings pointing to the _v1_4() transforms (C3)
- Detect native FOCUS 1.3/1.4 exports in x_SourceVersion and treat 1.4 as the current version for x_SourceChanges (H1)
- Always preserve removed ProviderName/PublisherName values in x_SourceValues (M1)
- Apply review suggestions: raw table annotation wording, FOCUS 1.2+ comment fixes, remove stale placement comments
- Split IngestionSetup_v1_4.kql into IngestionSetup_v1_4.kql (Prices + Costs) and IngestionSetup_v1_4_Datasets.kql (remaining datasets, alphabetized) to stay under the Bicep loadTextContent() 131,072-character limit; register the new file in app.bicep and .build.config

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@flanakin

Copy link
Copy Markdown
Collaborator Author

Pushed 30f926d addressing the round-2 review feedback and the remaining W2 fix list. Mapping of changes to review comments:

Applied

  • 3308931802Costs_raw annotation wording updated to (removed; keep for back-compat) for ProviderName/PublisherName/Region
  • 3309022571 / 3309026016 / 3309054996// FOCUS 1.3// FOCUS 1.2+ on the CommitmentDiscountQuantity, CommitmentDiscountUnit, x_AmortizationClass, and SkuPriceDetails fallbacks
  • 3309129381HostProviderName now falls back to deprecated ProviderName (suggestion applied as-is)
  • 3309175754ServiceProviderName now falls back to deprecated PublisherName, then ProviderName, then 'Microsoft'. Went one step beyond the suggestion because the earlier PublisherName fix-up can yield '' (when x_PublisherCategory isn't Cloud Provider), and FOCUS mandates non-null ServiceProviderName — the extra arms guarantee that
  • 3309171288 / 3309205246 — both "ProviderName and PublisherName removed…" placement comments deleted
  • 3309082327 — the new dataset sections are now alphabetized with BillingPeriods first. They landed in a new IngestionSetup_v1_4_Datasets.kql (BillingPeriods, CommitmentDiscountUsage, ContractCommitments, InvoiceDetails, Recommendations, Transactions) because IngestionSetup_v1_4.kql was ~2K characters from the Bicep loadTextContent() 131,072-character limit (PR Add x_RecommendationCategory and x_RecommendationSubcategory enrichments to v1_4 Recommendations #2194 already tripped it). Prices stays with Costs in IngestionSetup_v1_4.kql because Costs_transform_v1_4() joins Prices_final_v1_4 and the Costs_raw update policy validates that reference at deployment time — scripts in the hub-database.bicep map deploy as an unordered loop, so each file must be self-contained. Both files registered in app.bicep and .build.config.

Also in this push (W2 fix list)

  • C2: all 22 new Costs_raw columns projected from the C360 ActualCosts/AmortizedCosts transforms (v1_4, plus v1_0/v1_2 retrofits per the InvoiceId = '' convention) — verified programmatically against the raw table (166 columns, exact match ×6 transforms)
  • C3: all 7 v1_2 update policies disabled + deprecation docstrings pointing to _v1_4()
  • H1: x_SourceVersion infers '1.4' from ServiceProviderName/HostProviderName/InvoiceDetailId presence; x_SourceChanges now treats '1.4' as current (everything else labeled Legacy/UnknownFocusVersion)
  • M1: ProviderName/PublisherName always captured in x_SourceValues via the removed-column convention (checkString(..., old_X, ''))

Deliberately not done (tracked follow-ups — native-column research, #2159)

  • 3309107501 — back-porting CommitmentProgramEligibilityDetails from older exports (needs the CommitmentDiscountEligibility open dataset); left null for now
  • 3309131626 / 3309133481 — porting ContractCommitment* and InvoiceDetailId from older exports; no source data mapping exists yet, so inventing one here would be speculative

Verification: Build-Toolkit finops-hub green; both v1_4 files ≤ 110K (91,437 / 37,175 bytes); zero conflict markers; zero v1_3 identifiers; HubsIngestionQueries.Tests.ps1 192/192 passed.

@microsoft-github-policy-service microsoft-github-policy-service Bot added Needs: Review 👀 PR that is ready to be reviewed and removed Needs: Attention 👋 Issue or PR needs to be reviewed by the author or it will be closed due to no activity labels Jul 15, 2026
flanakin and others added 3 commits July 15, 2026 16:55
Resolved IngestionSetup_v1_2.kql docstring conflict by combining dev's
AmortizedCost copy-paste fix with the DEPRECATED annotation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ports #2151 (strcat-free emptiness checks), #2189 (max()-based price
lookup for duplicate on-demand rows), and the AmortizedCost docstring
copy-paste fix into IngestionSetup_v1_4.kql, which was copied from
v1_2 before those fixes landed on dev.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Renames 12 unprefixed columns (BenefitCategory, Created, Model, ...) to
their spec IDs (ContractCommitmentBenefitCategory, ...) and adds the two
missing columns (ContractCommitmentDescription, ServiceProviderName),
bringing the dataset to the full 30 FOCUS columns across the raw table,
parquet mapping, transform, and final table. Hub x_ add-on columns are
unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@flanakin

Copy link
Copy Markdown
Collaborator Author

Synced with dev and applied the Q4 schema fix:

  • Merge: merged flanakin/focus14 (now in sync with origin/dev). One conflict in IngestionSetup_v1_2.kql — dev's AmortizedCost docstring copy-paste fix vs the DEPRECATED annotation; resolved by combining both.
  • Ported dev fixes to v1.4: IngestionSetup_v1_4.kql was copied from v1_2 before [Hubs] Fix ingestion OOM errors by eliminating strcat() in emptiness checks #2151 and fix(hubs): use max() when duplicate on-demand price rows collapse under one lookup key #2189 landed. Ported the strcat-free emptiness checks, the max()-based on-demand price lookup, and the AmortizedCost docstring fix. Version-normalized diff vs v1_2 confirms the transform fix regions are at parity.
  • Q4 — ContractCommitments spec alignment: renamed 12 unprefixed columns to their FOCUS 1.4 spec IDs (e.g., BenefitCategoryContractCommitmentBenefitCategory) and added the 2 missing columns (ContractCommitmentDescription, ServiceProviderName), verified against the FOCUS 1.4 spec dataset definition — now the full 30 FOCUS columns across the raw table, parquet mapping, transform, and final table. Hub x_ add-on columns unchanged. The flattened ContractCommitment* per-row columns in the Costs tables are untouched (separate design).

Verified: Build-Toolkit finops-hub exit 0; v1_4 KQL files ≤ 110 KB (91.6 KB / 38.0 KB); no conflict markers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Needs: Review 👀 PR that is ready to be reviewed Tool: FinOps hubs Data pipeline solution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants