diff --git a/src/queries/INDEX.md b/src/queries/INDEX.md index e49f28e14..e15e09e73 100644 --- a/src/queries/INDEX.md +++ b/src/queries/INDEX.md @@ -2,7 +2,9 @@ > **Note:** Refer to the [FinOps hub database documentation](./finops-hub-database-guide.md) for table and column definitions. -> **Tip:** If you do not find a more specific or suitable query for your analysis, start with the [`costs-enriched-base`](./catalog/costs-enriched-base.kql) query. It provides the full enrichment and savings logic for all cost columns and is the recommended foundation for custom analytics and reporting. +This catalog contains 37 scenario-specific FinOps Hub KQL queries used by the FinOps Toolkit agents and the Azure SRE Agent recipe. + +> **Tip:** Prefer the narrowest scenario-specific query that answers the question. Use [`costs-enriched-base`](./catalog/costs-enriched-base.kql) when you need an enriched row-level baseline for scoped custom analysis or repeated drill-downs. --- @@ -10,14 +12,18 @@ | Common FinOps Task / Scenario | Recommended Query Name/ID | Description / When to Use | How to Use | | ------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- | -| View all available cost and enrichment columns for custom analysis | [costs-enriched-base](./catalog/costs-enriched-base.kql) | Use as a foundation for any custom analytics or reporting. | Set `startDate`/`endDate` to define analysis period; use as foundation for custom reports aligned with "Understand Cloud Usage & Cost" domain. | +| View all available cost and enrichment columns for scoped drill-down analysis | [costs-enriched-base](./catalog/costs-enriched-base.kql) | Use only when row-level enriched cost samples are required after narrower aggregate queries do not answer the question. | Set `startDate`/`endDate` to a narrow period; use for scoped row-level diagnostics aligned with the "Understand Usage & Cost" domain. | | Analyze monthly cost trends (e.g., for CIO/leadership reporting) | [monthly-cost-trend](./catalog/monthly-cost-trend.kql) | Track total billed and effective cost by month for trend analysis and executive reporting. | Set `startDate`/`endDate` for trend period; supports "Reporting and Analytics" capability for executive dashboards. | | Identify top cost-driving resource groups | [top-resource-groups-by-cost](./catalog/top-resource-groups-by-cost.kql) | Find which resource groups are responsible for the most spend in a given period. | Adjust `N` parameter (default: 5) for result count; set `startDate`/`endDate` for specific analysis periods. | | Report quarterly cost by resource group | [quarterly-cost-by-resource-group](./catalog/quarterly-cost-by-resource-group.kql) | Summarize effective cost by resource group for quarterly or multi-month reporting. | Set `startDate`/`endDate` for fiscal quarters; adjust `N` parameter (default: 5) for top resource groups shown. | -| Analyze cost by region for optimization | [cost-by-region-trend](./catalog/cost-by-region-trend.kql) | Break down effective cost by Azure region to identify regional cost drivers. | Set `startDate`/`endDate` for analysis period; supports "Workload Optimization" capability for region selection. | +| Analyze cost by region for optimization | [cost-by-region-trend](./catalog/cost-by-region-trend.kql) | Break down effective cost by Azure region to identify regional cost drivers. | Set `startDate`/`endDate` for analysis period; supports Usage Optimization and Architecting & Workload Placement for region selection. | | Identify top resource types by cost and usage | [top-resource-types-by-cost](./catalog/top-resource-types-by-cost.kql) | See which resource types (VMs, storage, etc.) are most used and most expensive. | Set `N` parameter (default: 10) for result count; set `startDate`/`endDate` for specific period analysis. | | Identify top services by cost | [top-services-by-cost](./catalog/top-services-by-cost.kql) | Find which Azure services (e.g., SQL, App Service) are driving the most cost. | Set `N` parameter (default: 10) for result count; set `startDate`/`endDate` for analysis period; key for cost visibility. | | Allocate cost by financial hierarchy (Billing Profile, Invoice Section, Team, Product, App) | [cost-by-financial-hierarchy](./catalog/cost-by-financial-hierarchy.kql) | For detailed allocation and showback/chargeback reporting. | Set `N` parameter (default: 5) for top results; set `startDate`/`endDate`; supports "Allocation" capability in FinOps framework. | +| Allocate Azure OpenAI cost by application, team, and environment | [ai-cost-by-application](./catalog/ai-cost-by-application.kql) | Break down Azure OpenAI costs by resource tags for showback, chargeback, and unit economics. | Set `startDate`/`endDate` for a complete month or analysis period; validate tag coverage before using results for chargeback. | +| Analyze daily Azure OpenAI cost and token trends | [ai-daily-trend](./catalog/ai-daily-trend.kql) | Track daily AI workload token consumption and effective cost. | Set `startDate`/`endDate` for the trend window; use for anomaly detection, forecasting, and run-rate discussions. | +| Compare Azure OpenAI model cost efficiency | [ai-model-cost-comparison](./catalog/ai-model-cost-comparison.kql) | Compare cost per 1K tokens, list cost, and discount percentage by model. | Set `startDate`/`endDate`; use with model selection, commitment, and rate optimization decisions. | +| Break down Azure OpenAI token usage by model and direction | [ai-token-usage-breakdown](./catalog/ai-token-usage-breakdown.kql) | Break token consumption and cost down by model and inferred input/output direction. | Set `startDate`/`endDate`; use for AI workload unit economics and prompt/output cost mix analysis. | | Benchmark service prices and savings (list, contracted, effective, negotiated, commitment) | [service-price-benchmarking](./catalog/service-price-benchmarking.kql) | Compare prices and savings by service for benchmarking and procurement. | Set `startDate`/`endDate` for benchmark period; supports "Benchmarking" capability in "Quantify Business Value" domain. | | Forecast future cloud spend | [cost-forecasting-model](./catalog/cost-forecasting-model.kql) | Project future costs for budgeting and planning. | Set `startDate`/`endDate` for historical data; adjust `forecastPeriods` (default: 90) and `interval` (default: 1d) for forecast horizon. | | Detect cost anomalies and spikes | [cost-anomaly-detection](./catalog/cost-anomaly-detection.kql) | Identify unusual cost spikes or drops for investigation. | Set `numberOfMonths` (default: 12) for lookback period; adjust `interval` (default: 1d) for detection granularity. | @@ -27,15 +33,30 @@ | List top commitment (RI/SP) transactions | [top-commitment-transactions](./catalog/top-commitment-transactions.kql) | Identify the largest reservation or savings plan purchases and their impact. | Set `N` parameter (default: 10) for result count; set `startDate`/`endDate` to analyze specific purchase periods. | | List top other (non-commitment, non-usage) transactions | [top-other-transactions](./catalog/top-other-transactions.kql) | Analyze large purchases not covered by RI/SP (e.g., support, marketplace, etc.). | Set `N` parameter (default: 10) for result count; set `startDate`/`endDate` to analyze non-usage spend. | | Analyze reservation recommendations and break-even points | [reservation-recommendation-breakdown](./catalog/reservation-recommendation-breakdown.kql) | Review Microsoft recommendations for new reservations and their projected savings. | Run without parameters for all recommendations; filter by service or region for targeted optimization. | - -> **Tip:** For any scenario not listed, start with [costs-enriched-base](./catalog/costs-enriched-base.kql) and customize as needed. +| Measure % of cost on resources with no tags (KPI: Untagged Costs) | [percentage-untagged-costs](./catalog/percentage-untagged-costs.kql) | FinOps Foundation KPI quantifying cost from resources that carry zero tags. | Set `startDate`/`endDate` for the KPI window; output column `UntaggedPercent` is ready for dashboards and SLO tracking. | +| Measure % of cost on resources lacking required allocation evidence (KPI: Unallocated Costs) | [percentage-unallocated-costs](./catalog/percentage-unallocated-costs.kql) | FinOps Foundation KPI quantifying cost that cannot be allocated due to missing rule, cost center, or ownership-tag evidence. | Edit `allocationEvidenceTagKeys` to your allocation policy; pair with `allocation-accuracy-index` for the complement view. | +| Measure Allocation Accuracy Index (AAI) | [allocation-accuracy-index](./catalog/allocation-accuracy-index.kql) | Directly attributed effective cost expressed as a 0-100 percentage — FinOps Foundation KPI. | Edit `allocationEvidenceTagKeys` to your allocation policy; use the same evidence set as `percentage-unallocated-costs` for a comparable view. | +| Measure tag policy compliance (KPI: Tagging Policy Compliance) | [tagging-policy-compliance](./catalog/tagging-policy-compliance.kql) | FinOps Foundation KPI measuring share of cost on resources satisfying mandatory tag keys. | Edit `requiredTagKeys` to your tagging policy; uses `mv-apply` to verify every required key is present. | +| Measure anomaly detection rate over a window (KPI: Anomaly Detection Rate) | [anomaly-detection-rate](./catalog/anomaly-detection-rate.kql) | FinOps Foundation KPI — percentage of effective spend on anomaly-flagged days via `series_decompose_anomalies`. | Set `startDate`/`endDate`; adjust `sensitivity` (default 1.5) for tolerance. | +| Measure total monetary variance for anomaly events (KPI: Total Unpredicted Variance of Spend) | [anomaly-variance-total](./catalog/anomaly-variance-total.kql) | FinOps Foundation KPI — sum of absolute monetary variance for detected anomaly events. | Set `numberOfMonths` and `interval`; result `TotalUnpredictedVariance` aggregates absolute residual cost. | +| Measure cost data visibility delay (KPI: Cost Visibility Delay) | [cost-visibility-delay](./catalog/cost-visibility-delay.kql) | FinOps Foundation KPI — percentile latency between `ChargePeriodEnd` and ingestion time. | Inspect `P50DelayHours`, `P90DelayHours`, `P99DelayHours`, and `MaxDelayHours` to monitor data-freshness SLOs. | +| Monitor cost data refresh cadence (KPI: Frequency of Data Updates) | [data-update-frequency](./catalog/data-update-frequency.kql) | FinOps Foundation KPI — last successful ingestion + inter-update gaps. | Inspect `LastUpdate`, `MedianIntervalHours`, `P90IntervalHours`, and `MaxIntervalHours` to validate the export schedule. | +| Measure commitment discount utilization (KPI: Commitment Utilization Score) | [commitment-utilization-score](./catalog/commitment-utilization-score.kql) | FinOps Foundation KPI — weighted ratio of utilized commitment quantity over purchased. | Set `startDate`/`endDate` covering full commitment periods; supports Rate Optimization domain. | +| Measure commitment discount waste (KPI: Commitment Discount Waste) | [commitment-discount-waste](./catalog/commitment-discount-waste.kql) | FinOps Foundation KPI — complement of utilization (unused commitment value / total). | Set `startDate`/`endDate` covering full commitment periods; use alongside `commitment-utilization-score` for both sides of the ledger. | +| Measure compute spend covered by commitments (KPI: Compute Spend Covered by Commitments) | [compute-spend-commitment-coverage](./catalog/compute-spend-commitment-coverage.kql) | FinOps Foundation KPI — share of compute `ContractedCost` backed by commitment discounts. | Set `startDate`/`endDate`; supports Rate Optimization and Commitment-Based Discount Management capabilities. | +| Measure effective and hourly compute cost per CPU core (KPI: Compute Cost per Core) | [compute-cost-per-core](./catalog/compute-cost-per-core.kql) | FinOps Foundation dual KPI — Effective Average and Hourly Cost per CPU Core for VM SKUs. | Set `startDate`/`endDate`; covers the canonical 3-step VM core-hour pattern. Pairs with `top-resource-types-by-cost` for rightsizing analysis. | +| Measure Cost Optimization Index (COIN) | [cost-optimization-index](./catalog/cost-optimization-index.kql) | FinOps Foundation KPI — accepted / total Microsoft cost recommendations. | Run without parameters; uses `Recommendations` table directly so no time window applies. | +| Measure cost per GB stored (KPI: Cost per Gigabytes Stored) | [cost-per-gb-stored](./catalog/cost-per-gb-stored.kql) | FinOps Foundation KPI — `BilledCost` over total GB-month for storage SKUs. | Set `startDate`/`endDate`; supports Usage Optimization for storage rightsizing. | +| Measure MACC consumption vs commitment drawdown (KPI: Consumption vs Commitment) | [macc-consumption-vs-commitment](./catalog/macc-consumption-vs-commitment.kql) | FinOps Foundation KPI — share of MACC commitment consumed in the current period. | Set `startDate`/`endDate` to align with a MACC term; supports Quantify Business Value reporting. | +| Measure storage tier distribution (KPI: Percent Storage on Frequent Access Tier) | [storage-tier-distribution](./catalog/storage-tier-distribution.kql) | FinOps Foundation KPI — distribution of storage cost across hot/cool/cold/archive tiers. | Set `startDate`/`endDate`; supports Usage Optimization decisions for tiering policy. | + +> **Tip:** For any scenario not listed, first look for a narrower aggregate query. Use [costs-enriched-base](./catalog/costs-enriched-base.kql) only when row-level enrichment is required for a scoped custom analysis. --- ## References -- [FinOps Framework (Microsoft Learn)](https://learn.microsoft.com/cloud-computing/finops/framework/finops-framework) +- [FinOps Framework (FinOps Foundation)](https://www.finops.org/framework/) - [Implementing FinOps Guide (Microsoft Learn)](https://learn.microsoft.com/cloud-computing/finops/implementing-finops-guide) - [Adopt FinOps on Azure (Microsoft Learn)](https://learn.microsoft.com/training/modules/adopt-finops-on-azure/) - [FinOps hub Database Documentation](./finops-hub-database-guide.md) -- [FinOps Foundation](https://www.finops.org/framework/) diff --git a/src/queries/KPI.md b/src/queries/KPI.md new file mode 100644 index 000000000..fe36fb4d3 --- /dev/null +++ b/src/queries/KPI.md @@ -0,0 +1,83 @@ +| KPI | Formula | Capabilities | Technology Categories | Source | Toolkit Queries | +|-----|---------|--------------|-----------------------|--------|-----------------| +| Total Migration Cost Savings | [(Current Cost on existing Infra (DC / Private Cloud / existing public CSP) – Effective Cost on the target platform (new public cloud CSP)] + [Shared Effective Costs associated with existing tools & people effort required to manage current infrastructure – Shared effective costs associated with target state tools & people effort required to manage target architecture] | | | https://www.finops.org/kpi/total-migration-cost-savings/ | | +| Fixed Percentage Apportionment Validation | Total apportionment of shared Effective Costs / Total shared Effective costs | Allocation | | https://www.finops.org/kpi/fixed-percentage-apportionment-validation/ | | +| Percentage of Costs Associated with Unallocated CSP Cloud Resources | (Total Effective Costs Associated w/Unallocated CSP Cloud Resources During a Period of Time / Total CSP Cloud Costs During a Period of Time) | Allocation | | https://www.finops.org/kpi/percentage-of-costs-associated-with-unallocated-csp-cloud-resources/ | [percentage-unallocated-costs](catalog/percentage-unallocated-costs.kql) | +| Percentage of Costs Associated with Untagged CSP Cloud Resources | (Total Effective Costs Associated with Untagged CSP Cloud Resources During a Period of Time / Total CSP Effective Cost During a Period of Time) x 100 | Allocation | | https://www.finops.org/kpi/percentage-of-costs-associated-with-untagged-csp-cloud-resources/ | [percentage-untagged-costs](catalog/percentage-untagged-costs.kql) | +| Percentage of CSP Cloud Costs that are Tagging Policy Compliant | (Total Effective Costs Associated with Tagging Policy Compliant CSP Cloud Resources During a Period of Time / Total CSP Cloud Effective Costs During a Period of Time) x 100 | Allocation | | https://www.finops.org/kpi/percentage-of-csp-cloud-costs-that-are-tagging-policy-compliant/ | [tagging-policy-compliance](catalog/tagging-policy-compliance.kql) | +| Percentage of Unallocated Shared CSP Cloud Cost | Percentage of Unallocated Shared CSP Effective Cloud Cost = (Unallocated Shared CSP Effective Cloud Cost/Total CSP Effective Cloud Cost) | Allocation | | https://www.finops.org/kpi/percentage-of-unallocated-shared-csp-cloud-cost/ | | +| Usage or Spend Apportionment Validation | Total apportionment of Shared Costs / Total shared costs | Allocation | | https://www.finops.org/kpi/usage-or-spend-apportionment-validation/ | | +| Data Center Management Efficiency | Operational Load Factor = (FTEs × Hourly Rate) / (Managed Infrastructure Value) | Allocation; Data Ingestion; Reporting & Analytics; Unit Economics | | https://www.finops.org/kpi/data-center-management-efficiency/ | | +| Total Cost of Ownership per Workload | TCO per Workload = (Total Costs Over Lifecycle) / (Number of Workloads) | Allocation; Reporting & Analytics; Unit Economics | | https://www.finops.org/kpi/total-cost-of-ownership-per-workload/ | | +| Allocation Accuracy Index (AAI) | Allocation Accuracy Index (AAI) = (Directly Attributed Costs / Total Infrastructure Costs) × 100 | Allocation; Reporting & Analytics; Usage Optimization | | https://www.finops.org/kpi/allocation-accuracy-index-aai/ | [allocation-accuracy-index](catalog/allocation-accuracy-index.kql) | +| Anomaly-Detected Cost Avoidance | Total cost of the unpredicted & correctable amount of spend * Projected Amount of Days the anomaly could have occurred if the underlying issue was not addressed | Anomaly Management | | https://www.finops.org/kpi/anomaly-detected-cost-avoidance/ | | +| Total Unpredicted Variance of Spend | Total effective cost associated with all anomalies events detected less the predicted spend of the services related to the identified anomalies. | Anomaly Management | | https://www.finops.org/kpi/total-unpredicted-variance-of-spend/ | [anomaly-variance-total](catalog/anomaly-variance-total.kql) | +| Anomaly Detection Rate | Total Cost of Anomaly Spikes / Total AI Spend = Anomaly Cost % where (adjust for your needs): Green ( Yellow (2-7%): Warning. Minor anomaly trend Red (> 7%): Critical. You have a “runaway” costs. | Anomaly Management; Data Ingestion; Reporting & Analytics | | https://www.finops.org/kpi/anomaly-detection-rate/ | [anomaly-detection-rate](catalog/anomaly-detection-rate.kql) | +| Computational Waste Percentage | ((Unit Consumed by Failed Jobs + Idle Time + Technical Spillage) / Total Compute Units Consumed) x 100 | Anomaly Management; Reporting & Analytics; Usage Optimization | | https://www.finops.org/kpi/computational-waste-percentage/ | | +| Effective Scan Efficiency | Effective Scan Efficiency = (Units of Data Scanned / Total Units of Data in Table) x 100 | Anomaly Management; Usage Optimization | | https://www.finops.org/kpi/effective-scan-efficiency/ | | +| SaaS Optimization ROI | Optimization ROI = Savings from SaaS Optimization Actions / Implementation Cost | Architecting & Workload Placement; Data Ingestion; Licensing & SaaS; Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/saas-optimization-roi/ | | +| Redundant Application Coverage Percent | Redundant Application Coverage Percent = Spend on Overlapping Tools / Total SaaS Spend | Architecting & Workload Placement; Data Ingestion; Licensing & SaaS; Reporting & Analytics; Usage Optimization | | https://www.finops.org/kpi/redundant-application-coverage-percent/ | | +| CSP Cloud Budget Burn Rate | (Actual Cloud Spending / Time Period) | Budgeting | | https://www.finops.org/kpi/csp-cloud-budget-burn-rate/ | [monthly-cost-trend](catalog/monthly-cost-trend.kql) | +| Percentage Variance of Budgeted vs. Actual CSP Cloud Spend | ((Budgeted CSP Cloud Spend – Actual CSP Cloud Spend) / Budgeted CSP Cloud Spend) | Budgeting | | https://www.finops.org/kpi/percentage-variance-of-budgeted-vs-actual-csp-cloud-spend/ | | +| Percentage Variance of Budgeted vs. Forecasted CSP Cloud Spend | ((Budgeted CSP Effective Cost for a Time Period – Forecasted CSP Effective Cost for that Time Period) / Budgeted CSP Effective Cost for that Time Period) | Budgeting | | https://www.finops.org/kpi/percentage-variance-of-budgeted-vs-forecasted-csp-cloud-spend/ | | +| CSP Cloud Carbon Budget Burn Rate | (Allocated Cloud Emissions / Time Period) | Budgeting; Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/csp-cloud-carbon-budget-burn-rate/ | | +| Cost Visibility Delay | Cost Visibility Delay = Time of Displaying Cost – Time of Cost Generation | Data Ingestion | | https://www.finops.org/kpi/cost-visibility-delay/ | [cost-visibility-delay](catalog/cost-visibility-delay.kql) | +| ETL Processing Time | Processing Time = End Time – Start Time | Data Ingestion | | https://www.finops.org/kpi/etl-processing-time/ | | +| Frequency of Data Updates | Data Update Frequency = Time of Latest Cost Update – Time of Previous Cost Update | Data Ingestion | | https://www.finops.org/kpi/frequency-of-data-updates/ | [data-update-frequency](catalog/data-update-frequency.kql) | +| Consumption versus Commitment | Consumption versus Commitment = Actual Consumption Units / Committed Units | Data Ingestion; Forecasting; Licensing & SaaS; Rate Optimization; Reporting & Analytics | | https://www.finops.org/kpi/consumption-versus-commitment/ | [macc-consumption-vs-commitment](catalog/macc-consumption-vs-commitment.kql) | +| Feature Adoption Rate | Feature Adoption Rate = (Users Using Premium Features / Premium Licenses Assigned) x 100 | Data Ingestion; Licensing & SaaS; Rate Optimization; Reporting & Analytics | | https://www.finops.org/kpi/feature-adoption-rate/ | | +| License Utilisation Rate | License Utilisation Rate = (Assigned Licenses / Purchased Licenses) x 100 | Data Ingestion; Licensing & SaaS; Rate Optimization; Reporting & Analytics | | https://www.finops.org/kpi/license-utilisation-rate/ | | +| Active-to-Provisioned User Ratio | Active-to-Provisioned User Ratio = (Active Users / Provisioned Users) x 100 | Data Ingestion; Licensing & SaaS; Reporting & Analytics; Usage Optimization | Licences, SaaS | https://www.finops.org/kpi/active-to-provisioned-user-ratio/ | | +| Data Center Power Usage Effectiveness | Power Usage Effectiveness (PUE) = Total Facility Power / IT Equipment Power | Data Ingestion; Reporting & Analytics; Sustainability | | https://www.finops.org/kpi/data-center-power-usage-effectiveness/ | | +| Cost per Inference | Cost Per Inference = Total Inference Costs / Number of Inference Requests​ | Data Ingestion; Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/cost-per-inference/ | | +| Resource Utilization Efficiency | Resource Utilization Efficiency = Actual Resource Utilization / Provisioned Capacity​ | Data Ingestion; Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/resource-utilization-efficiency/ | | +| Token Consumption Metrics | Cost Per Token = Total Cost / Number of Tokens Used | Data Ingestion; Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/token-consumption-metrics/ | [ai-token-usage-breakdown](catalog/ai-token-usage-breakdown.kql), [ai-daily-trend](catalog/ai-daily-trend.kql), [ai-model-cost-comparison](catalog/ai-model-cost-comparison.kql) | +| Training Cost Efficiency | Training Cost Efficiency = Training Costs / Performance Metric | Data Ingestion; Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/training-cost-efficiency/ | | +| Count of FinOps Learning Opportunities Offered | Target number of courses grouped by time period | FinOps Education & Enablement | | https://www.finops.org/kpi/count-of-finops-learning-opportunities-offered-monthly-quarterly-annually/ | | +| Percentage of FinOps Personas Who Have Completed Training | (Target for certification – actual count certified ) / target for certification * 100 | FinOps Education & Enablement | | https://www.finops.org/kpi/percentage-of-stakeholders-all-finops-personas-who-have-completed-finops-training/ | | +| Forecast Accuracy Rate (Spend) | ((Forecasted Public Cloud Spend – Actual Public Cloud Spend) / Forecasted Public Cloud Spend) | Forecasting | | https://www.finops.org/kpi/forecast-accuracy-rate-spend/ | | +| Forecast Drift Rate | ((New Forecasted Cloud Infrastructure Spend – Previous Historic Forecasted Cloud Infrastructure Spend) / Previous Forecasted Cloud Infrastructure Spend) | Forecasting | | https://www.finops.org/kpi/forecast-drift-rate/ | | +| Forecast (Present) Actual Rate (Carbon) | ((Forecasted Public Cloud emissions – Actual Public Cloud emissions) / Forecasted Public Cloud emissions) | Forecasting; Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/forecast-present-actual-rate-carbon/ | | +| Time to Achieve Business Value | Time to Value (days) = Total Value associated with AI Service / daily Cost of Alternative solution | Forecasting; Planning & Estimating; Reporting & Analytics; Unit Economics | | https://www.finops.org/kpi/time-to-achieve-business-value/ | | +| Forecast Accuracy Rate (Usage) | Formula : For specific ServiceName or ServiceCategory or SKU ((Forecasted Resource Utilization – Actual Resource Utilization) / Forecasted Resource Utilization) | Forecasting; Sustainability | | https://www.finops.org/kpi/forecast-accuracy-rate-usage/ | | +| SaaS Unit Cost | SaaS Unit Cost = Total SaaS Cost / Total Units of Consumption | Intersecting Disciplines; Licensing & SaaS; Reporting & Analytics; Usage Optimization | | https://www.finops.org/kpi/saas-unit-cost/ | | +| Anomaly-Detected Carbon Avoidance | (Total emissions of the unpredicted & correctable amount of emissions) x (Projected Amount of Days the anomaly could have occurred if the underlying issue was not addressed) | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/anomaly-detected-carbon-avoidance/ | | +| Carbon Efficient Locations | (Count of active resources in low carbon region) / (total active resources) | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/carbon-efficient-locations/ | | +| Carbon per [Time] Compute Category | Compute CO2e per Time unit = Compute CO2e / Time Unit | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/carbon-per-time-compute-category/ | | +| Carbon per [Time] Storage Category | Storage CO2e per Time unit = Storage (CO2e/gigabytes) / Time unit | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/carbon-per-time-storage-category/ | | +| Carbon Waste / Carbon Efficiency | Carbon Waste Formula: (Carbon of currently allocated resource) – (Carbon of optimised resource for utilisation need) Carbon Efficiency Formula: (Carbon of optimized resource for utilization need) / (Carbon of currently allocated resource) | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/carbon-waste-carbon-efficiency/ | | +| Forecast Accuracy Rate for Carbon Emissions | ((Forecasted Public Cloud emissions – Actual Public Cloud emissions) / Forecasted Public Cloud emissions) | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/forecast-accuracy-rate-for-carbon-emissions/ | | +| Optimizing Electricity PUE, WUE, Between Regions, or Cloud Service Providers | PUE = Total Facility Energy / IT Equipment Energy WUE = Total Water Used by the Data Center / Total IT Equipment Energy Consumption (in kWh) | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/optimizing-electricity-pue-wue-between-regions-or-cloud-service-providers/ | | +| Percentage of Carbon Associated with Untagged CSP Cloud Resources | (Total emissions Associated with Untagged CSP Cloud Resources During a Period of Time / Total CSP Cloud emissions During a Period of Time) x 100 | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/percentage-of-carbon-associated-with-untagged-csp-cloud-resources/ | | +| Percentage of CSP Cloud Carbon that is Tagging Policy Compliant | (Total emissions Associated with Tagging Policy Compliant CSP Cloud Resources During a Period of Time / Total CSP Cloud Emissions During a Period of Time) x 100 | Intersecting Disciplines; Sustainability | | https://www.finops.org/kpi/percentage-of-csp-cloud-carbon-that-is-tagging-policy-compliant/ | | +| Carbon per Unit of Cloud Service SKU | Carbon per unit (kgCO2e) = Total Carbon Emissions (kgCO2e) / Total Resource Usage (SKU) | Intersecting Disciplines; Sustainability; Unit Economics | | https://www.finops.org/kpi/carbon-per-unit-of-cloud-service-sku/ | | +| Carbon per Unit of Cloud Spend | CO2e/unit of currency = Effective Cost / CO2e | Intersecting Disciplines; Sustainability; Unit Economics | | https://www.finops.org/kpi/carbon-per-unit-of-cloud-spend/ | | +| General Ledger Recharge Rate | (Total CSP Cloud Spend recorded as a Journal in the General Ledger) / (Total CSP* cloud spend for the month) | Invoicing & Chargeback | | https://www.finops.org/kpi/general-ledger-recharge-rate/ | | +| General Ledger Recharge Rate per Cost Center | (Total CSP* Cloud Spend per Cost Center recorded as a Journal in the General Ledger) / (Total CSP* cloud spend per Cost center for the month) | Invoicing & Chargeback | | https://www.finops.org/kpi/general-ledger-recharge-rate-per-cost-center/ | | +| Cloud Spend Percentage of Recurring Revenue | (Monthly total cloud invoice amount / Monthly Recurring Revenue) and/or (Annual total cloud invoice amount / Annual Recurring Revenue) | KPIs & Benchmarking; Planning & Estimating; Reporting & Analytics; Unit Economics | | https://www.finops.org/kpi/cloud-spend-percentage-of-recurring-revenue/ | | +| Value for AI Initiatives | Return On Investment = (Financial Benefits – Costs) / Costs * 100 | KPIs & Benchmarking; Reporting & Analytics; Unit Economics | AI | https://www.finops.org/kpi/value-for-ai-initiatives/ | | +| Data Value Density | Data Value Density = Total Business Revenue or Value Index / Total Data Platform TCO | Licensing & SaaS; Rate Optimization; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/data-value-density/ | | +| Time to First Prompt | Time to First Prompt = Deployment Date – Start Date of Initiative Development | Planning & Estimating; Reporting & Analytics | | https://www.finops.org/kpi/time-to-first-prompt/ | | +| Commitment Utilization Score | Commitment Utilization Score = (Used Commitment / Total Commitment) x 100 | Rate Optimization | | https://www.finops.org/kpi/commitment-utilization-score/ | [commitment-utilization-score](catalog/commitment-utilization-score.kql) | +| Effective Average Compute Cost per Core | (Effective Cost + Unused Commitment Discount Cost + Compute Cost) / Total number of Cores | Rate Optimization | | https://www.finops.org/kpi/effective-average-compute-cost-per-core/ | [compute-cost-per-core](catalog/compute-cost-per-core.kql) | +| Effective Savings Rate Percentage | Option 1: (CB Discount Savings – Cost to achieve CB Discount Savings) / Compute On-Demand Equivalent Spend Option 2: 1 – (Actual Spend with Discounts / Equivalent Spend at On Demand Rate) | Rate Optimization | | https://www.finops.org/kpi/effective-savings-rate-percentage/ | [savings-summary-report](catalog/savings-summary-report.kql) | +| Percent of Compute Spend Covered by Commitment Discounts | Compute Cost after Commitment Discount / On-Demand Compute Cost | Rate Optimization | | https://www.finops.org/kpi/percent-of-compute-spend-covered-by-commitment-based-discounts/ | [compute-spend-commitment-coverage](catalog/compute-spend-commitment-coverage.kql) | +| Percentage of Commitment Discount Waste | (Cost of Commitment Discount unused / total cost Commitment Discount) x 100 | Rate Optimization | | https://www.finops.org/kpi/percentage-of-commitment-based-discount-waste/ | [commitment-discount-waste](catalog/commitment-discount-waste.kql) | +| Cost Optimization Index (COIN) | COIN Score = [1 – (Total Savings Opportunity / Total Cost)] * 100 The resulting score from 0-100 serves as an objective benchmark for cost efficiency. Total Cost is based on the aggregate cost of any relevant scope of infrastructure and measured in any desired currency. Total Savings Opportunity is based on the sum of individual Savings Opportunities. Savings Opportunities are usage patterns within that scope of infrastructure which indicate expected areas of inefficiency and waste. These are calculated as projected savings in the same currency as Total Cost. Each Savings Opportunity will need its own cost model to identify potential savings. This scoring system can help: Identify areas to save money within run rate Compare teams, services, org or organizations cost efficiency Identify priority savings opportunities to target across the broader organization | Rate Optimization; Usage Optimization | | https://www.finops.org/kpi/cost-optimization-index-coin/ | [cost-optimization-index](catalog/cost-optimization-index.kql) | +| Pipeline Jobs Waste | Pipeline Jobs Waste = (Failed\|\|Error\|\|Cancelled\|\|Timed_out) Job Cost / Total Job Cost | Rate Optimization; Usage Optimization | | https://www.finops.org/kpi/pipeline-jobs-waste/ | | +| Storage Decay Ratio | Storage Decay Ratio = (Volume of Unaccessed Data / Total Data Volume) x 100 | Rate Optimization; Usage Optimization | | https://www.finops.org/kpi/storage-decay-ratio/ | | +| CPI – Cost Performance Indicator | CPI = Earned Value / Effective Cost at time period Where, Earned Value = Budgeted Cost per selected time period * % Complete And where, % Complete = Expected Unit Economic per selected time period / Actual Unit Economic per selected time period * 100 | Reporting & Analytics | | https://www.finops.org/kpi/cpi-cost-performance-indicator/ | | +| Number of Monthly Active Users (MAU) for Cost Insight Products | Number = Raw count of Monthly Active User (MAU), or any one single user accessing any insight Cost Insight Products | Reporting & Analytics | | https://www.finops.org/kpi/number-of-monthly-active-users-mau-for-cost-insight-products/ | | +| Percentage of Normalized, Queryable Data within Repository | Spend included in queryable data warehouse (or similar) / Total Cloud Spend | Reporting & Analytics | | https://www.finops.org/kpi/percentage-of-normalized-queryable-data-within-repository/ | | +| Cost per API Call | Cost Per API Call = Total API Costs / Number of API Calls | Reporting & Analytics; Unit Economics | | https://www.finops.org/kpi/cost-per-api-call/ | | +| Efficiency & Performance ROI | Optimization ROI = (Cost Savings + Performance Gains) / Implementation Cost | Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/efficiency-performance-roi/ | | +| IT Lifecycle Waste Efficiency | Efficiency KPI = ($ Potential Savings from Identified Waste) / (Total IT Cost in Scope) | Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/it-lifecycle-waste-efficiency/ | | +| Unified Cost & Usage Visibility | Reporting & Analytics Integration Completeness = (Integrated Usage & Cost Sources) / (Total Usage & Cost Sources) × 100 | Reporting & Analytics; Unit Economics; Usage Optimization | | https://www.finops.org/kpi/unified-cost-usage-visibility/ | | +| Auto-scaling Efficiency Rate | Auto-scaling efficiency rate = Maximum capacity cost of running workload to meet workload demand / Cost of running workload with auto-scaling to meet same workload demand. The higher the efficiency rate the more effective the auto-scaling is. Effective Cost can be used in this formula, or the List Cost metric can be used to eliminate the effect of discounts and focus entirely on the scaling effect. | Sustainability; Usage Optimization | | https://www.finops.org/kpi/auto-scaling-efficiency-rate/ | | +| Percent of Unused Resources | Volumes: Pricing Quantity of Unattached Volumes / Total Number of Volumes Number of Snapshots not accessed over a defined period of time / total # of Snapshots EIPS: Quantity of unattached Elastic IPs / Total Elastic IP’s | Sustainability; Usage Optimization | | https://www.finops.org/kpi/percent-of-unused-resources/ | | +| Percent Storage on Frequent Access Tier | Number of GB in Standard (or “frequently accessed” tiers vs. total GBs stored) | Sustainability; Usage Optimization | | https://www.finops.org/kpi/percent-storage-on-frequent-access-tier/ | [storage-tier-distribution](catalog/storage-tier-distribution.kql) | +| Percentage of Legacy Resource | Number of instances on ‘legacy’ resource types / total number of instances (can be done by app, account, total) | Sustainability; Usage Optimization | | https://www.finops.org/kpi/percentage-of-legacy-resource/ | | +| Percentage Resource Utilization | Compute: CPU Utilization Rate (total consumed quantity in core-hrs or cpu-hrs / Total CPUs Allocated), Memory Utilization Rate (Total Usage in GB / Total Memory Allocated) Volumes: Throughput Utilized / Throughput Provisioned, IOPS Utilized / IOPS provisioned, Storage Utilized / Storage Provisioned (per volume) | Sustainability; Usage Optimization | | https://www.finops.org/kpi/percentage-resource-utilization/ | | +| Power Schedule Adherence Rate | Power schedule adherence percentage per month = (Total number of hours the system is required to run/Total number of hours the system is running) | Sustainability; Usage Optimization | | https://www.finops.org/kpi/power-schedule-adherence-rate/ | | +| Cost per Gigabytes Stored | Cloud Storage Costs / Number of GB stored | Unit Economics | | https://www.finops.org/kpi/cost-per-gigabytes-stored/ | [cost-per-gb-stored](catalog/cost-per-gb-stored.kql) | +| Hourly Cost per CPU Core | Hourly Cost / Number of CPU cores | Unit Economics | | https://www.finops.org/kpi/hourly-cost-per-cpu-core/ | [compute-cost-per-core](catalog/compute-cost-per-core.kql) | diff --git a/src/queries/catalog/ai-cost-by-application.kql b/src/queries/catalog/ai-cost-by-application.kql new file mode 100644 index 000000000..915223cbd --- /dev/null +++ b/src/queries/catalog/ai-cost-by-application.kql @@ -0,0 +1,35 @@ +// ============================================================================ +// Query: Azure OpenAI Cost by Application +// Description: +// Breaks down Azure OpenAI costs by application, cost center, team, and environment tags. +// Useful for AI workload showback, chargeback, and unit economics analysis. +// Author: FinOps Toolkit Team +// Parameters: +// startDate: Start date for the reporting period (e.g., startofmonth(ago(30d))) +// endDate: End date for the reporting period (e.g., startofmonth(now())) +// Output: +// Each row represents a tagged Azure OpenAI cost grouping with token count and cost metrics. +// Usage: +// Use this query to allocate Azure OpenAI usage and cost to applications, teams, and environments. +// Last Updated: 2026-05-26 +// ============================================================================ + +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where x_SkuMeterSubcategory has "OpenAI" +| extend Application = tostring(Tags["application"]) +| extend CostCenter = case( + isnotempty(tostring(Tags["cost-center"])), tostring(Tags["cost-center"]), + isnotempty(tostring(Tags["CostCenter"])), tostring(Tags["CostCenter"]), + "") +| extend Environment = tostring(Tags["environment"]) +| extend Team = tostring(Tags["team"]) +| summarize + TokenCount = sum(ConsumedQuantity), + EffectiveCost = sum(EffectiveCost), + BilledCost = sum(BilledCost) + by BillingCurrency, Application, CostCenter, Team, Environment, ResourceName, x_SkuMeterSubcategory +| extend CostPer1KTokens = iff(TokenCount == 0, 0.0, EffectiveCost / TokenCount * 1000) +| order by EffectiveCost desc diff --git a/src/queries/catalog/ai-daily-trend.kql b/src/queries/catalog/ai-daily-trend.kql new file mode 100644 index 000000000..fb0e3067f --- /dev/null +++ b/src/queries/catalog/ai-daily-trend.kql @@ -0,0 +1,27 @@ +// ============================================================================ +// Query: Azure OpenAI Daily Cost and Token Trend +// Description: +// Returns daily Azure OpenAI token consumption and effective cost. +// Useful for AI workload anomaly detection, forecasting, and trend reporting. +// Author: FinOps Toolkit Team +// Parameters: +// startDate: Start date for the reporting period (e.g., ago(30d)) +// endDate: End date for the reporting period (e.g., now()) +// Output: +// Each row represents one day with token count, cost, and cost per 1K tokens. +// Usage: +// Use this query to monitor AI workload consumption trends and detect daily cost spikes. +// Last Updated: 2026-05-26 +// ============================================================================ + +let startDate = ago(30d); +let endDate = now(); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where x_SkuMeterSubcategory has "OpenAI" +| summarize + DailyTokens = sum(ConsumedQuantity), + DailyCost = sum(EffectiveCost) + by Day = bin(ChargePeriodStart, 1d), BillingCurrency +| extend CostPer1KTokens = iff(DailyTokens == 0, 0.0, DailyCost / DailyTokens * 1000) +| order by Day asc, BillingCurrency asc diff --git a/src/queries/catalog/ai-model-cost-comparison.kql b/src/queries/catalog/ai-model-cost-comparison.kql new file mode 100644 index 000000000..12d13a977 --- /dev/null +++ b/src/queries/catalog/ai-model-cost-comparison.kql @@ -0,0 +1,31 @@ +// ============================================================================ +// Query: Azure OpenAI Model Cost Comparison +// Description: +// Compares token volume, effective cost, list cost, and discount percentage by model. +// Useful for AI model cost efficiency analysis and rate optimization. +// Author: FinOps Toolkit Team +// Parameters: +// startDate: Start date for the reporting period (e.g., startofmonth(ago(30d))) +// endDate: End date for the reporting period (e.g., startofmonth(now())) +// Output: +// Each row represents one Azure OpenAI model or SKU description with cost per 1K tokens. +// Usage: +// Use this query to compare model economics and identify where model or commitment changes may reduce AI spend. +// Last Updated: 2026-05-26 +// ============================================================================ + +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where x_SkuMeterSubcategory has "OpenAI" +| extend Model = x_SkuDescription +| summarize + TokenCount = sum(ConsumedQuantity), + EffectiveCost = sum(EffectiveCost), + ListCost = sum(ListCost) + by BillingCurrency, Model +| extend CostPer1KTokens = iff(TokenCount == 0, 0.0, EffectiveCost / TokenCount * 1000) +| extend ListPer1KTokens = iff(TokenCount == 0, 0.0, ListCost / TokenCount * 1000) +| extend DiscountPercent = iff(ListCost == 0, 0.0, (ListCost - EffectiveCost) / ListCost * 100) +| order by EffectiveCost desc diff --git a/src/queries/catalog/ai-token-usage-breakdown.kql b/src/queries/catalog/ai-token-usage-breakdown.kql new file mode 100644 index 000000000..25cccecfe --- /dev/null +++ b/src/queries/catalog/ai-token-usage-breakdown.kql @@ -0,0 +1,35 @@ +// ============================================================================ +// Query: Azure OpenAI Token Usage Breakdown +// Description: +// Breaks Azure OpenAI token consumption down by model version and input/output direction. +// Calculates effective unit cost per token and cost per 1K tokens. +// Author: FinOps Toolkit Team +// Parameters: +// startDate: Start date for the reporting period (e.g., startofmonth(ago(30d))) +// endDate: End date for the reporting period (e.g., startofmonth(now())) +// Output: +// Each row represents one model and direction with token count and cost metrics. +// Usage: +// Use this query to analyze AI workload unit economics, token direction mix, and model cost efficiency. +// Last Updated: 2026-05-26 +// ============================================================================ + +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where x_SkuMeterSubcategory has "OpenAI" +| extend Model = x_SkuDescription +| extend Direction = case( + Model contains "Input", "Input", + Model contains "Output", "Output", + "Other") +| summarize + TokenCount = sum(ConsumedQuantity), + EffectiveCost = sum(EffectiveCost), + BilledCost = sum(BilledCost), + ListCost = sum(ListCost) + by BillingCurrency, Model, Direction +| extend UnitCostPerToken = iff(TokenCount == 0, 0.0, EffectiveCost / TokenCount) +| extend CostPer1KTokens = UnitCostPerToken * 1000 +| order by EffectiveCost desc diff --git a/src/queries/catalog/allocation-accuracy-index.kql b/src/queries/catalog/allocation-accuracy-index.kql new file mode 100644 index 000000000..8e1793df2 --- /dev/null +++ b/src/queries/catalog/allocation-accuracy-index.kql @@ -0,0 +1,36 @@ +// ============================================================================ +// Query: Allocation Accuracy Index +// Description: +// Calculates the percentage of total effective cost that is directly attributed using a +// three-signal heuristic: allocation rule, cost center, or ownership-tag evidence. +// KPI: Allocation Accuracy Index (AAI) +// Formula: Allocation Accuracy Index (AAI) = (Directly Attributed Costs / Total Infrastructure Costs) × 100 +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start date for the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; end date for the reporting period (default: startofmonth(now())) +// allocationEvidenceTagKeys: dynamic; tag keys treated as ownership evidence (default: dynamic(['cost-center','team','owner','application','product'])) +// Output: +// Each row represents one BillingCurrency and returns DirectlyAttributedCost, TotalEffectiveCost, +// and AAI for Hub-visible CSP costs in the reporting window. +// Usage: +// Use this query to measure how much effective cost is directly attributable within FinOps Hub. +// Scope Notes: Hub-only AAI. Does not include on-prem, SaaS, or other infrastructure outside Hub schema. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let allocationEvidenceTagKeys = dynamic(['cost-center','team','owner','application','product']); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| extend tmp_TagKeys = coalesce(bag_keys(Tags), dynamic([])) +| extend tmp_IsAttributed = isnotempty(x_CostAllocationRuleName) + or isnotempty(x_CostCenter) + or array_length(set_intersect(tmp_TagKeys, allocationEvidenceTagKeys)) > 0 +| summarize + DirectlyAttributedCost = todouble(sumif(EffectiveCost, tmp_IsAttributed)), + TotalEffectiveCost = todouble(sum(EffectiveCost)) + by BillingCurrency +| extend AAI = iff(TotalEffectiveCost == 0, 0.0, DirectlyAttributedCost / TotalEffectiveCost * 100.0) +| project BillingCurrency, DirectlyAttributedCost, TotalEffectiveCost, AAI diff --git a/src/queries/catalog/anomaly-detection-rate.kql b/src/queries/catalog/anomaly-detection-rate.kql new file mode 100644 index 000000000..975f33b99 --- /dev/null +++ b/src/queries/catalog/anomaly-detection-rate.kql @@ -0,0 +1,53 @@ +// ============================================================================ +// Query: Anomaly Detection Rate +// Description: +// Calculates the percentage of effective spend attributable to anomaly-flagged days. +// Builds daily cost series per service category and billing currency, then applies time-series anomaly detection. +// KPI: Anomaly Detection Rate +// Formula: Total Cost of Anomaly Spikes / Total Spend = Anomaly Cost % +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; Start date for the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; End date for the reporting period (default: startofmonth(now())) +// sensitivity: real; Anomaly detection sensitivity for series_decompose_anomalies() (default: 1.5) +// Output: +// Each row returns a BillingCurrency and ServiceCategory pair (plus an Overall rollup row) with AnomalyCost, TotalCost, and AnomalyRatePercent as double values. +// Usage: +// Use this query to quantify how much effective spend falls on anomaly-flagged days by service category and by billing currency. +// Scope Notes: +// Treats both positive spikes and negative drops as anomalies (AnomalyFlags != 0); the Overall row is per BillingCurrency only. +// Missing days are zero-filled via make-series default=0.0, and commitment purchase rows are excluded to avoid amortization double-counting. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 13 rows returned (per service category) +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let sensitivity = 1.5; +let expanded = materialize( + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) + | summarize DailyCost = sum(todouble(EffectiveCost)) + by bin(ChargePeriodStart, 1d), ServiceCategory, BillingCurrency + | make-series CostSeries = sum(DailyCost) default=0.0 + on ChargePeriodStart from startDate to endDate step 1d + by ServiceCategory, BillingCurrency + | extend AnomalyFlags = series_decompose_anomalies(CostSeries, sensitivity) + | mv-expand CostSeries to typeof(double), AnomalyFlags to typeof(int) +); +let perCategory = + expanded + | summarize + AnomalyCost = todouble(sumif(CostSeries, AnomalyFlags != 0)), + TotalCost = todouble(sum(CostSeries)) + by BillingCurrency, ServiceCategory + | extend AnomalyRatePercent = todouble(iff(TotalCost == 0.0, 0.0, AnomalyCost / TotalCost * 100.0)); +let overall = + expanded + | summarize + AnomalyCost = todouble(sumif(CostSeries, AnomalyFlags != 0)), + TotalCost = todouble(sum(CostSeries)) + by BillingCurrency + | extend ServiceCategory = 'Overall' + | extend AnomalyRatePercent = todouble(iff(TotalCost == 0.0, 0.0, AnomalyCost / TotalCost * 100.0)); +union perCategory, overall +| project BillingCurrency, ServiceCategory, AnomalyCost, TotalCost, AnomalyRatePercent diff --git a/src/queries/catalog/anomaly-variance-total.kql b/src/queries/catalog/anomaly-variance-total.kql new file mode 100644 index 000000000..5c2625851 --- /dev/null +++ b/src/queries/catalog/anomaly-variance-total.kql @@ -0,0 +1,65 @@ +// ============================================================================ +// Query: Total Unpredicted Variance of Spend +// Description: +// Calculates the net unpredicted variance between actual effective cost and the anomaly baseline +// for anomaly-flagged daily buckets by service category and billing currency. +// KPI: Total Unpredicted Variance of Spend +// Formula: Total effective cost associated with all anomaly events detected less the predicted spend of the services related to the identified anomalies. +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start of the reporting period (default: startofmonth(ago(12 * 30d))) +// endDate: datetime; end of the reporting period (default: now()) +// anomalyThreshold: real; anomaly detection sensitivity threshold (default: 1.5) +// Output: +// Each row returns BillingCurrency, ServiceCategory, AnomalyEventCount, UnpredictedVarianceSigned, +// and UnpredictedVarianceAbs for anomaly-detected daily buckets; "(All Services)" rows provide +// per-currency totals across all service categories. +// Usage: +// Use this query to quantify net anomalous overspend or underspend by service category and currency. +// Scope Notes: +// Per-group anomaly detection runs independently per (ServiceCategory, BillingCurrency), and groups +// with sparse history are filtered when no anomaly baseline is produced. Each BillingCurrency is +// reported separately and must not be summed across currencies without FX conversion. The default +// 12-month window supports STL seasonality decomposition, and UnpredictedVarianceAbs is defined as +// abs(sum(actual - baseline)) rather than sum(abs(actual - baseline)). +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 13 rows returned (per service category) +// ========================================================================= +let startDate = startofmonth(ago(12 * 30d)); +let endDate = now(); +let anomalyThreshold = 1.5; +let anomalyBuckets = + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) + | summarize DailyCost = sum(EffectiveCost) by ServiceCategory, BillingCurrency, bin(ChargePeriodStart, 1d) + | make-series CostSeries = sum(DailyCost) default=0.0 on ChargePeriodStart from startDate to endDate step 1d by ServiceCategory, BillingCurrency + | extend (ad_flag, ad_score, ad_baseline) = series_decompose_anomalies(CostSeries, anomalyThreshold) + | mv-expand ChargePeriodStart to typeof(datetime), CostSeries to typeof(real), ad_flag to typeof(real), ad_score to typeof(real), ad_baseline to typeof(real) + | extend ad_flag = toint(ad_flag), CostSeries = toreal(CostSeries), ad_score = toreal(ad_score), ad_baseline = toreal(ad_baseline) + | where isnotnull(ad_baseline) + | where ad_flag != 0 + | extend Variance = todouble(CostSeries) - todouble(ad_baseline); +union +( + anomalyBuckets + | summarize + AnomalyEventCount = count(), + UnpredictedVarianceSigned = todouble(sum(Variance)), + UnpredictedVarianceAbs = todouble(abs(sum(Variance))) + by BillingCurrency, ServiceCategory +), +( + anomalyBuckets + | summarize + AnomalyEventCount = count(), + UnpredictedVarianceSigned = todouble(sum(Variance)), + UnpredictedVarianceAbs = todouble(abs(sum(Variance))) + by BillingCurrency + | extend ServiceCategory = "(All Services)" +) +| project + BillingCurrency, + ServiceCategory, + AnomalyEventCount, + UnpredictedVarianceSigned = todouble(UnpredictedVarianceSigned), + UnpredictedVarianceAbs = todouble(UnpredictedVarianceAbs) diff --git a/src/queries/catalog/commitment-discount-utilization.kql b/src/queries/catalog/commitment-discount-utilization.kql index ddba80284..bcefcc39d 100644 --- a/src/queries/catalog/commitment-discount-utilization.kql +++ b/src/queries/catalog/commitment-discount-utilization.kql @@ -20,10 +20,10 @@ let endDate = startofmonth(now()); let base = Costs() | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate | extend x_SkuCoreCount = toint(coalesce(x_SkuDetails.VCPUs, x_SkuDetails.vCores, 0)) -| extend x_ConsumedCoreHours = iff(isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, todecimal('')); -let total = base | summarize Total=todecimal(sum(x_ConsumedCoreHours)); +| extend x_ConsumedCoreHours = iff(isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, toreal('')); +let total = base | summarize Total=toreal(sum(x_ConsumedCoreHours)); base -| summarize TotalConsumedCoreHours = todecimal(sum(x_ConsumedCoreHours)) by CommitmentDiscountType +| summarize TotalConsumedCoreHours = toreal(sum(x_ConsumedCoreHours)) by CommitmentDiscountType | extend CommitmentDiscountType = iff(isempty(CommitmentDiscountType), 'On Demand', CommitmentDiscountType) | extend PercentOfTotal = 100.0 * TotalConsumedCoreHours / toscalar(total) | project CommitmentDiscountType, TotalConsumedCoreHours=todouble(TotalConsumedCoreHours), PercentOfTotal=todouble(PercentOfTotal) diff --git a/src/queries/catalog/commitment-discount-waste.kql b/src/queries/catalog/commitment-discount-waste.kql new file mode 100644 index 000000000..68d65c3dc --- /dev/null +++ b/src/queries/catalog/commitment-discount-waste.kql @@ -0,0 +1,88 @@ +// ============================================================================ +// Query: Percentage of Commitment Discount Waste +// Description: +// Computes the percentage of commitment discount waste for each commitment by +// comparing unused amortized EffectiveCost to total commitment EffectiveCost. +// Includes a grand-total row per BillingCurrency across all commitments in the +// reporting window. +// KPI: Percentage of Commitment Discount Waste +// Formula: (Cost of Commitment Discount unused / total cost Commitment Discount) x 100 +// Author: FinOps toolkit +// Parameters: +// startDate: datetime = startofmonth(ago(30d)) +// endDate: datetime = startofmonth(now()) +// Output: +// BillingCurrency: string. Billing currency for the commitment or grand-total row. +// CommitmentDiscountId: string. Commitment identifier; empty string on the grand-total row. +// CommitmentDiscountName: string. Commitment name; '(All Commitments)' on the grand-total row. +// CommitmentDiscountType: string. Commitment type; empty string on the grand-total row. +// UnusedCost: double. Sum of EffectiveCost for rows where CommitmentDiscountStatus == 'Unused'. +// TotalCost: double. Sum of EffectiveCost across all commitment rows after filters. +// WastePercent: double. Percentage of TotalCost represented by UnusedCost. +// Usage: +// Use this query to identify underutilized commitment discounts and quantify waste by commitment and billing currency. +// Scope Notes: +// - Double-counting prevention: This query operates on amortized-style rows only +// (purchase rows excluded). +// - Unused row semantics: CommitmentDiscountStatus == 'Unused' rows represent +// wasted amortized commitment cost for the charge period. +// - CommitmentDiscountStatus values: Only 'Used' and 'Unused' are expected. +// - Currency mixing: Grand-total output is scoped per BillingCurrency; cross- +// currency aggregation is out of scope. +// - Period window: Default startDate/endDate cover one calendar month; adjust +// the let bindings for rolling windows. +// - KPI limitation vs canonical definition: This implementation uses +// EffectiveCost as the cost basis for commitment-discount waste analytics. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS_SCHEMA_WITH_DATA_GAP — query executed and returned expected columns; the hub has 0 commitment-discount rows in the test window, so value semantics require re-UAT on a commitment-active hub. +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let filteredCosts = + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where isnotempty(CommitmentDiscountId) + | where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)); +let commitmentWaste = + filteredCosts + | summarize + UnusedCost = sum(iff(CommitmentDiscountStatus == 'Unused', EffectiveCost, real(0))), + TotalCost = sum(EffectiveCost) + by BillingCurrency, CommitmentDiscountId, CommitmentDiscountName, CommitmentDiscountType + | extend WastePercent = iff(TotalCost == 0.0, 0.0, todouble(UnusedCost) / todouble(TotalCost) * 100.0) + | project + BillingCurrency, + CommitmentDiscountId, + CommitmentDiscountName, + CommitmentDiscountType, + UnusedCost = todouble(UnusedCost), + TotalCost = todouble(TotalCost), + WastePercent = todouble(WastePercent); +let grandTotals = + filteredCosts + | summarize + UnusedCost = sum(iff(CommitmentDiscountStatus == 'Unused', EffectiveCost, real(0))), + TotalCost = sum(EffectiveCost) + by BillingCurrency + | extend + CommitmentDiscountId = '', + CommitmentDiscountName = '(All Commitments)', + CommitmentDiscountType = '' + | extend WastePercent = iff(TotalCost == 0.0, 0.0, todouble(UnusedCost) / todouble(TotalCost) * 100.0) + | project + BillingCurrency, + CommitmentDiscountId, + CommitmentDiscountName, + CommitmentDiscountType, + UnusedCost = todouble(UnusedCost), + TotalCost = todouble(TotalCost), + WastePercent = todouble(WastePercent); +union commitmentWaste, grandTotals +| order by BillingCurrency asc, WastePercent desc, CommitmentDiscountName asc +| project + BillingCurrency, + CommitmentDiscountId, + CommitmentDiscountName, + CommitmentDiscountType, + UnusedCost, + TotalCost, + WastePercent diff --git a/src/queries/catalog/commitment-utilization-score.kql b/src/queries/catalog/commitment-utilization-score.kql new file mode 100644 index 000000000..63cd0dcfe --- /dev/null +++ b/src/queries/catalog/commitment-utilization-score.kql @@ -0,0 +1,67 @@ +// ============================================================================ +// Query: Commitment Utilization Score +// Description: +// Computes commitment utilization by commitment discount ID over the reporting window. +// Returns per-commitment and per-currency grand-total utilization amount, potential, and score. +// KPI: Commitment Utilization Score +// Formula: Commitment Utilization Score = (Used Commitment / Total Commitment) x 100 +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start date for the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; end date for the reporting period (default: startofmonth(now())) +// Output: +// Each row represents one commitment or a per-currency grand total with BillingCurrency, +// CommitmentDiscountId, CommitmentDiscountName, CommitmentDiscountCategory, +// CommitmentDiscountType, UtilizationAmount, UtilizationPotential, and UtilizationScore. +// Usage: +// Use this query to monitor how fully Azure commitment discounts were utilized during the reporting window. +// Scope Notes: +// Microsoft Azure commitments only; score is window-based, grand totals are per BillingCurrency, +// and Usage-category quantities remain ConsumedUnit-dependent but valid as a ratio within each commitment. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS_SCHEMA_WITH_DATA_GAP — query executed and returned expected columns; the hub has 0 commitment-discount rows in the test window, so value semantics require re-UAT on a commitment-active hub. +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let commitmentRows = materialize( + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where isnotempty(CommitmentDiscountId) + | extend x_CommitmentDiscountUtilizationPotential = case( + ChargeCategory == 'Purchase', toreal(0), + ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), toreal(EffectiveCost), + CommitmentDiscountCategory == 'Usage', toreal(ConsumedQuantity), + CommitmentDiscountCategory == 'Spend', toreal(EffectiveCost), + toreal(0)) + | extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, toreal(0)) +); +let commitmentSummary = + commitmentRows + | summarize + UtilizationAmount = todouble(sum(x_CommitmentDiscountUtilizationAmount)), + UtilizationPotential = todouble(sum(x_CommitmentDiscountUtilizationPotential)) + by BillingCurrency, CommitmentDiscountId, CommitmentDiscountName, CommitmentDiscountCategory, CommitmentDiscountType + | extend UtilizationScore = iff(UtilizationPotential > 0, todouble(UtilizationAmount / UtilizationPotential) * 100.0, todouble(0)); +union + commitmentSummary, + ( + commitmentSummary + | summarize + UtilizationAmount = todouble(sum(UtilizationAmount)), + UtilizationPotential = todouble(sum(UtilizationPotential)) + by BillingCurrency + | extend + CommitmentDiscountId = '(All)', + CommitmentDiscountName = '(Grand Total)', + CommitmentDiscountCategory = '', + CommitmentDiscountType = '', + UtilizationScore = iff(UtilizationPotential > 0, todouble(UtilizationAmount / UtilizationPotential) * 100.0, todouble(0)) + ) +| project + BillingCurrency, + CommitmentDiscountId, + CommitmentDiscountName, + CommitmentDiscountCategory, + CommitmentDiscountType, + UtilizationAmount, + UtilizationPotential, + UtilizationScore diff --git a/src/queries/catalog/compute-cost-per-core.kql b/src/queries/catalog/compute-cost-per-core.kql new file mode 100644 index 000000000..fb9335927 --- /dev/null +++ b/src/queries/catalog/compute-cost-per-core.kql @@ -0,0 +1,43 @@ +// ============================================================================ +// Query: Effective Average Compute Cost per Core and Hourly Cost per CPU Core +// Description: +// Computes two compute-efficiency KPIs from FinOps Hub cost records in one pass over Costs(). +// It summarizes compute usage cost, unused commitment cost, and computed core-hours by billing currency. +// KPI: Effective Average Compute Cost per Core; Hourly Cost per CPU Core +// Formula: +// HourlyCostPerCore = sum(EffectiveCost for compute usage rows) / sum(x_ConsumedCoreHours for compute usage rows) +// EffectiveAverageCostPerCore = (sum(EffectiveCost for compute usage rows) + sum(EffectiveCost where CommitmentDiscountStatus=='Unused' AND isnotempty(CommitmentDiscountCategory))) / sum(x_ConsumedCoreHours for compute usage rows) +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start of the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; end of the reporting period, exclusive (default: startofmonth(now())) +// Output: +// One row per BillingCurrency with BillingCurrency, ComputeUsageEffectiveCost, UnusedCommitmentCost, TotalCoreHours, HourlyCostPerCore, and EffectiveAverageCostPerCore. +// Usage: +// Use this query to compare amortized compute spend per consumed vCPU-hour with and without unused commitment cost. +// Scope Notes: +// Azure-only VM usage detection uses x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') with ChargeCategory == 'Usage'. +// UnusedCommitmentCost is restricted to unused compute commitments (CommitmentDiscountStatus == 'Unused' AND the +// commitment SKU is itself in the compute meter categories), so non-compute commitment waste does not inflate +// compute unit economics. Wave-boundary fix (T-3000.12 cross-model review): gpt-5.5 flagged that the prior +// un-scoped predicate mixed non-compute waste into the compute denominator. +// Core counts reflect billed VCPUs/vCores from x_SkuDetails, and all monetary values remain denominated in BillingCurrency rather than normalized USD. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned; HourlyCostPerCore=0.019 in BillingCurrency +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| extend tmp_IsVMUsage = x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') and ChargeCategory == 'Usage' +| extend tmp_IsComputeCommitment = x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') +| extend x_SkuCoreCount = iff(tmp_IsVMUsage, toint(coalesce(x_SkuDetails.VCPUs, x_SkuDetails.vCores)), toint('')) +| extend x_ConsumedCoreHours = iff(tmp_IsVMUsage and isnotempty(x_SkuCoreCount), toreal(x_SkuCoreCount * ConsumedQuantity), toreal('')) +| summarize + ComputeUsageEffectiveCost = todouble(sumif(EffectiveCost, tmp_IsVMUsage)), + UnusedCommitmentCost = todouble(sumif(EffectiveCost, CommitmentDiscountStatus == 'Unused' and isnotempty(CommitmentDiscountCategory) and tmp_IsComputeCommitment)), + TotalCoreHours = todouble(sum(x_ConsumedCoreHours)) + by BillingCurrency +| extend HourlyCostPerCore = todouble(iff(isnull(TotalCoreHours) or TotalCoreHours == 0.0, 0.0, ComputeUsageEffectiveCost / TotalCoreHours)) +| extend EffectiveAverageCostPerCore = todouble(iff(isnull(TotalCoreHours) or TotalCoreHours == 0.0, 0.0, (ComputeUsageEffectiveCost + UnusedCommitmentCost) / TotalCoreHours)) +| project BillingCurrency, ComputeUsageEffectiveCost, UnusedCommitmentCost, TotalCoreHours, HourlyCostPerCore, EffectiveAverageCostPerCore diff --git a/src/queries/catalog/compute-spend-commitment-coverage.kql b/src/queries/catalog/compute-spend-commitment-coverage.kql new file mode 100644 index 000000000..2c7f25939 --- /dev/null +++ b/src/queries/catalog/compute-spend-commitment-coverage.kql @@ -0,0 +1,32 @@ +// ============================================================================ +// Query: Percent of Compute Spend Covered by Commitment Discounts +// Description: +// Computes the share of compute spend covered by commitment discounts over a selected reporting window. +// Uses effective cost for commitment-backed compute spend and contracted cost for the compute baseline. +// KPI: Percent of Compute Spend Covered by Commitment Discounts +// Formula: Compute Cost after Commitment Discount / On-Demand Compute Cost +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start of the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; end of the reporting period, exclusive (default: startofmonth(now())) +// Output: +// One row per BillingCurrency with BillingCurrency, CommittedComputeEffectiveCost, TotalComputeContractedCost, and CoveragePercent. +// Usage: +// Use this query to measure how much compute spend is covered by commitment discounts without mixing currencies. +// Scope Notes: +// Denominator uses ContractedCost as the post-negotiated, pre-commitment baseline rather than public retail pricing. +// Scope is limited to ServiceCategory == 'Compute', excludes commitment purchase rows, and groups results by BillingCurrency. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned (CoveragePercent=0; expected given commitment data gap) +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| where ServiceCategory == 'Compute' +| summarize + CommittedComputeEffectiveCost = todouble(sumif(EffectiveCost, isnotempty(CommitmentDiscountCategory))), + TotalComputeContractedCost = todouble(sum(ContractedCost)) + by BillingCurrency +| extend CoveragePercent = todouble(iff(TotalComputeContractedCost == 0.0, 0.0, 100.0 * CommittedComputeEffectiveCost / TotalComputeContractedCost)) +| project BillingCurrency, CommittedComputeEffectiveCost, TotalComputeContractedCost, CoveragePercent diff --git a/src/queries/catalog/cost-optimization-index.kql b/src/queries/catalog/cost-optimization-index.kql new file mode 100644 index 000000000..d4a69c77b --- /dev/null +++ b/src/queries/catalog/cost-optimization-index.kql @@ -0,0 +1,51 @@ +// ============================================================================ +// Query: Cost Optimization Index (COIN) +// Description: +// Calculates the Cost Optimization Index by comparing total reservation recommendation savings opportunity +// against amortized total cost for the reporting window. Reported as a single hub-wide score because the +// Hub `Recommendations()` table does not carry a `BillingCurrency` dimension and savings cannot be safely +// attributed back to a specific currency. +// KPI: Cost Optimization Index (COIN) +// Formula: COIN Score = [1 – (Total Savings Opportunity / Total Cost)] * 100 +// Author: FinOps toolkit +// Parameters: +// startDate: datetime = startofmonth(ago(30d)) +// endDate: datetime = startofmonth(now()) +// Output: +// ReportingCurrencyScope: string ("HubWide") — single-row scope flag; see Scope Notes for multi-currency hubs. +// TotalSavingsOpportunity: double total RI/SP savings opportunity from Recommendations() for the reporting window. +// TotalCost: double amortized effective cost from Costs() for the reporting window, excluding commitment purchases. +// COINScore: double Cost Optimization Index score clamped to 0 when total cost is 0 or the computed score is negative. +// Usage: +// Use this query to estimate how optimized current spend is relative to reservation and savings plan opportunities in FinOps Hub. +// Scope Notes: +// COIN as computed here reflects ONLY commitment-discount (RI/SP) savings opportunities from Microsoft Azure Advisor. +// Does NOT include rightsizing, idle resources, or workload optimization. +// Wave-boundary fix (T-3000.13 cross-model review): gpt-5.5 flagged that grouping by `BillingCurrency` repeated the +// single global savings scalar against each currency's cost, producing misleading per-currency COIN values for +// multi-currency hubs. Recommendations() has no BillingCurrency column, so we now report one hub-wide row. For +// multi-currency hubs operators should normalize Costs() to a single reporting currency before consuming COINScore. +// Time-grain fix (post-UAT, 2026-05-28): Recommendations() has NO RELIABLE BUSINESS TIME GRAIN. The +// `x_RecommendationDate` column is documented but commonly null in live Hubs (verified null on every row in trey-hub +// 2026-05-28 UAT), and `x_IngestionTime` is the ingestion timestamp, not an applicability period. Reservation +// recommendations are point-in-time optimization snapshots, not dated events, so we aggregate the entire current +// recommendation set and compare against the windowed Cost amortization. Filtering Recommendations() by a date +// column would silently drop all unstamped rows (false COIN=100). Cost denominator remains windowed. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned (COIN≈97.66 with 18,976.30 reservation-recommendation savings opportunity in the hub billing currency, captured after time-grain fix) +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let totalSavingsOpportunity = toscalar( + Recommendations() + | where x_SourceType == 'ReservationRecommendations' + | summarize TotalSavingsOpportunity = todouble(sum(x_EffectiveCostSavings)) +); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| summarize TotalCost = todouble(sum(EffectiveCost)) +| extend ReportingCurrencyScope = 'HubWide' +| extend TotalSavingsOpportunity = todouble(coalesce(totalSavingsOpportunity, 0.0)) +| extend RawCOINScore = iff(TotalCost == 0.0, 0.0, (1.0 - TotalSavingsOpportunity / TotalCost) * 100.0) +| extend COINScore = todouble(iff(RawCOINScore < 0.0, 0.0, RawCOINScore)) +| project ReportingCurrencyScope, TotalSavingsOpportunity = todouble(TotalSavingsOpportunity), TotalCost = todouble(TotalCost), COINScore diff --git a/src/queries/catalog/cost-per-gb-stored.kql b/src/queries/catalog/cost-per-gb-stored.kql new file mode 100644 index 000000000..100df5cab --- /dev/null +++ b/src/queries/catalog/cost-per-gb-stored.kql @@ -0,0 +1,42 @@ +// ============================================================================ +// Query: Cost Per Gigabytes Stored +// Description: +// Computes the period-average storage cost per normalized GB-month for storage usage rows. +// It summarizes amortized storage cost and normalized storage consumption by billing currency. +// KPI: Cost per Gigabytes Stored +// Formula: Cloud Storage Costs / Number of GB stored +// Author: FinOps toolkit +// Parameters: +// startDate: datetime (default: startofmonth(ago(30d))) - Start date for the reporting period +// endDate: datetime (default: startofmonth(now())) - End date for the reporting period +// Output: +// Each row returns BillingCurrency, TotalStorageCost, TotalGBMonths, and CostPerGBMonth for qualifying storage usage in the reporting window. +// Usage: +// Use this query to measure the average amortized storage cost rate per GB-month over a reporting period. +// Scope Notes: ConsumedQuantity for storage represents GB-months over the billed period (time-weighted average), not point-in-time GB at rest. CostPerGBMonth is a period-average rate. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where ServiceCategory == 'Storage' and ChargeCategory == 'Usage' +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| extend x_NormalizedGB = case( + ConsumedUnit endswith 'TB', todouble(ConsumedQuantity) * 1024.0, + ConsumedUnit endswith 'PB', todouble(ConsumedQuantity) * 1048576.0, + ConsumedUnit endswith 'MB', todouble(ConsumedQuantity) / 1024.0, + todouble(ConsumedQuantity) +) +| extend ConsumedUnitBucket = case( + ConsumedUnit endswith 'TB', 'TB→GB (×1024)', + ConsumedUnit endswith 'PB', 'PB→GB (×1048576)', + ConsumedUnit endswith 'MB', 'MB→GB (÷1024)', + 'GB (pass-through)' +) +| summarize + TotalStorageCost = todouble(sum(EffectiveCost)), + TotalGBMonths = todouble(sum(x_NormalizedGB)) + by BillingCurrency +| extend CostPerGBMonth = todouble(iff(TotalGBMonths == 0.0 or isnull(TotalGBMonths), todouble(0), TotalStorageCost / TotalGBMonths)) +| project BillingCurrency, TotalStorageCost, TotalGBMonths, CostPerGBMonth diff --git a/src/queries/catalog/cost-visibility-delay.kql b/src/queries/catalog/cost-visibility-delay.kql new file mode 100644 index 000000000..4229011dc --- /dev/null +++ b/src/queries/catalog/cost-visibility-delay.kql @@ -0,0 +1,63 @@ +// ============================================================================ +// Query: Cost Visibility Delay +// Description: +// Computes the delay between when cost data was generated and when it became visible in the FinOps Hub. +// Aggregates median, p90, p99, and max delay by billing account and reporting month in hours and days. +// KPI: Cost Visibility Delay +// Formula: Cost Visibility Delay = Time of Displaying Cost – Time of Cost Generation +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start date for the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; end date for the reporting period (default: startofmonth(now())) +// Output: +// BillingAccountId: Billing account identifier for the aggregated bucket. +// ReportingMonth: startofmonth(ChargePeriodStart) reporting bucket. +// RowCount: Count of rows included after filtering out NULL x_IngestionTime values. +// P50DelayHours: Median visibility delay in hours. +// P90DelayHours: 90th percentile visibility delay in hours. +// P99DelayHours: 99th percentile visibility delay in hours. +// MaxDelayHours: Maximum visibility delay in hours. +// P50DelayDays: Median visibility delay in days. +// P90DelayDays: 90th percentile visibility delay in days. +// P99DelayDays: 99th percentile visibility delay in days. +// MaxDelayDays: Maximum visibility delay in days. +// Usage: +// Use this query to measure how quickly cost records become visible in the Hub after the charge period closes. +// Scope Notes: +// BillingCurrency is listed in the spec inputs but omitted from the required output grouping, so results aggregate by BillingAccountId and ReportingMonth only. +// Negative delays are retained to surface anomalies where ingestion precedes ChargePeriodEnd, and legacy hubs with NULL x_IngestionTime return no rows after filtering. +// This KPI is satisfied entirely from Costs() with no cross-table joins and no commitment-purchase exclusion. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS_WITH_NOTE — 1 row returned; 11 cols returned (7 spec-required + 4 documented *DelayDays extras per Output header) +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where isnotnull(x_IngestionTime) +| extend BillingAccountId = tolower(BillingAccountId) +| extend ReportingMonth = startofmonth(ChargePeriodStart) +| extend DelayHours = todouble((x_IngestionTime - ChargePeriodEnd) / 1h) +| summarize + RowCount = count(), + P50DelayHours = todouble(percentile(DelayHours, 50)), + P90DelayHours = todouble(percentile(DelayHours, 90)), + P99DelayHours = todouble(percentile(DelayHours, 99)), + MaxDelayHours = todouble(max(DelayHours)) + by BillingAccountId, ReportingMonth +| extend + P50DelayDays = todouble(P50DelayHours / 24.0), + P90DelayDays = todouble(P90DelayHours / 24.0), + P99DelayDays = todouble(P99DelayHours / 24.0), + MaxDelayDays = todouble(MaxDelayHours / 24.0) +| project + BillingAccountId, + ReportingMonth, + RowCount, + P50DelayHours, + P90DelayHours, + P99DelayHours, + MaxDelayHours, + P50DelayDays, + P90DelayDays, + P99DelayDays, + MaxDelayDays diff --git a/src/queries/catalog/data-update-frequency.kql b/src/queries/catalog/data-update-frequency.kql new file mode 100644 index 000000000..e22e400c3 --- /dev/null +++ b/src/queries/catalog/data-update-frequency.kql @@ -0,0 +1,36 @@ +// ============================================================================ +// Query: Frequency of Data Updates +// Description: +// Computes the cadence of Hub cost-data ingestion updates for each billing account +// by measuring the gaps between consecutive distinct x_IngestionTime values. +// KPI: Frequency of Data Updates +// Formula: Data Update Frequency = Time of Latest Cost Update – Time of Previous Cost Update +// Author: FinOps toolkit +// Parameters: +// None. This query intentionally uses the full available ingestion history from Costs(). +// Output: +// Each row represents one billing account with its latest ingestion timestamp, distinct +// update count, and median, P90, and maximum update intervals in hours. +// Usage: +// Use this query to measure ingestion freshness cadence and identify stale billing accounts. +// Scope Notes: +// Measures Hub ingestion cadence from x_IngestionTime, not the timing of underlying charges. +// Accounts with only one distinct ingestion timestamp return null interval statistics. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned +// ========================================================================= +Costs() +| where isnotnull(x_IngestionTime) +| extend BillingAccountId = tolower(BillingAccountId) +| summarize by BillingAccountId, x_IngestionTime +| sort by BillingAccountId asc, x_IngestionTime asc +| serialize +| extend prev_account = prev(BillingAccountId), prev_t = prev(x_IngestionTime) +| extend interval_hours = iff(BillingAccountId == prev_account, todouble((x_IngestionTime - prev_t) / 1h), real(null)) +| summarize + LastUpdate = max(x_IngestionTime), + UpdateCount = count(), + MedianIntervalHours = todouble(percentile(interval_hours, 50)), + P90IntervalHours = todouble(percentile(interval_hours, 90)), + MaxIntervalHours = todouble(max(interval_hours)) + by BillingAccountId +| project BillingAccountId, LastUpdate, UpdateCount, MedianIntervalHours, P90IntervalHours, MaxIntervalHours diff --git a/src/queries/catalog/macc-consumption-vs-commitment.kql b/src/queries/catalog/macc-consumption-vs-commitment.kql new file mode 100644 index 000000000..e61df18a0 --- /dev/null +++ b/src/queries/catalog/macc-consumption-vs-commitment.kql @@ -0,0 +1,61 @@ +// ============================================================================ +// Query: Consumption Versus Commitment +// Description: +// Calculates monthly actual consumption versus monetary commitment for Microsoft Azure Consumption Commitment (MACC) billing accounts. +// Uses EffectiveCost from Costs() as the consumption numerator and x_MonetaryCommitment from Transactions() as the commitment denominator. +// KPI: Consumption versus Commitment +// Formula: Consumption versus Commitment = Actual Consumption Units / Committed Units +// Author: FinOps toolkit +// Parameters: +// startDate: datetime = startofmonth(ago(30d)) +// endDate: datetime = startofmonth(now()) +// Output: +// Each row represents one billing profile + currency + calendar month with ConsumptionAmount, CommitmentAmount, ConsumptionVsCommitmentRatio, and CommitmentBurnPercent. Per-profile CommitmentAmount and ConsumptionAmount sum to account-level totals; the ratio is uniform within an account by construction of the proportional allocation. +// Usage: +// Use this query to monitor monthly MACC burn against committed spend for billing accounts in the active FinOps Hub. +// Scope Notes: +// MACC-only: Query targets Microsoft Azure Consumption Commitment (MACC) customers. Non-MACC billing accounts will join to zero CommitmentAmount and emit a ratio of 0.0. +// OPEN: x_MonetaryCommitment aggregation semantics need validation against a MACC-active FinOps Hub; if Transactions() repeats balances per billing period, replace sum() with max() or a deduplicated latest-event scan. +// Per-profile commitment allocation: x_BillingProfileId is unavailable in Transactions(), so the commitment is +// collected at the BillingAccountId level and then allocated PROPORTIONALLY across profiles by each profile's +// share of account consumption in the reporting window. This preserves the spec output contract (BillingProfileId +// in output) while keeping per-profile ratios analytically meaningful: the sum of CommitmentAmount across all +// profiles in an account equals the account-level commitment, and each profile's ratio represents that profile's +// share of MACC drawdown. Wave-boundary fix (T-3000.15 cross-model review): opus-4.6 flagged BillingAccountId in +// output (spec deviation); gpt-5.5 flagged that naive broadcasting would duplicate commitment per profile. The +// allocation pattern below resolves both findings. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS_SCHEMA_WITH_DATA_GAP — 1 row returned; test hub has 0 Transactions() rows and 0 x_MonetaryCommitment rows, so MACC value semantics require re-UAT on a MACC-active hub. +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let profileConsumption = + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) + | extend BillingAccountId = tolower(BillingAccountId) + | extend ReportingMonth = startofmonth(ChargePeriodStart) + | summarize ProfileConsumption = todouble(sum(EffectiveCost)) by BillingCurrency, BillingAccountId, x_BillingProfileId, ReportingMonth; +let accountCommitment = + Transactions() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where x_TransactionType in ('MACC', 'MonetaryCommitment', 'Microsoft Azure Consumption Commitment') + or isnotnull(x_MonetaryCommitment) + | extend BillingAccountId = tolower(BillingAccountId) + | extend ReportingMonth = startofmonth(ChargePeriodStart) + // VALIDATE on a MACC-active hub: sum(x_MonetaryCommitment) represents the total MACC obligation for the window. + // If the Hub deployment records this repeatedly per billing period instead of once per purchase, + // switch to max() or scan for the most-recent MACC event per period. + | summarize AccountCommitment = todouble(sum(x_MonetaryCommitment)) by BillingCurrency, BillingAccountId, ReportingMonth; +profileConsumption +| join kind=leftouter ( + profileConsumption + | summarize AccountConsumption = sum(ProfileConsumption) by BillingCurrency, BillingAccountId, ReportingMonth +) on BillingCurrency, BillingAccountId, ReportingMonth +| join kind=leftouter accountCommitment on BillingCurrency, BillingAccountId, ReportingMonth +| extend AccountCommitment = todouble(coalesce(AccountCommitment, 0.0)) +| extend ProfileShare = iff(AccountConsumption == 0.0, 0.0, ProfileConsumption / AccountConsumption) +| extend ConsumptionAmount = ProfileConsumption +| extend CommitmentAmount = AccountCommitment * ProfileShare +| extend ConsumptionVsCommitmentRatio = iff(CommitmentAmount == 0.0, 0.0, ConsumptionAmount / CommitmentAmount) +| extend CommitmentBurnPercent = ConsumptionVsCommitmentRatio * 100.0 +| project BillingCurrency, BillingProfileId = x_BillingProfileId, ReportingMonth, ConsumptionAmount, CommitmentAmount, ConsumptionVsCommitmentRatio, CommitmentBurnPercent diff --git a/src/queries/catalog/percentage-unallocated-costs.kql b/src/queries/catalog/percentage-unallocated-costs.kql new file mode 100644 index 000000000..84a6e1984 --- /dev/null +++ b/src/queries/catalog/percentage-unallocated-costs.kql @@ -0,0 +1,42 @@ +// ============================================================================ +// Query: Percentage of Unallocated Costs +// Description: +// Computes the share of EffectiveCost with no allocation evidence in Costs() using a +// heuristic: no cost allocation rule, no cost center, and no ownership tag key match. +// KPI: Percentage of Costs Associated with Unallocated CSP Cloud Resources +// Formula: (Total Effective Costs Associated with Unallocated CSP Cloud Resources During a Period of Time / Total CSP Cloud Costs During a Period of Time) +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; start of the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; end of the reporting period (default: startofmonth(now())) +// allocationEvidenceTagKeys: dynamic; tag keys that count as allocation evidence +// (default: dynamic(['cost-center','team','owner','application','product'])) +// Output: +// BillingCurrency: billing currency for the summarized row. +// UnallocatedEffectiveCost: total EffectiveCost for rows classified as unallocated. +// TotalEffectiveCost: total EffectiveCost for all rows after commitment-purchase exclusion. +// UnallocatedPercent: unallocated share on a 0-100 scale. +// Usage: +// Use this query to estimate how much spend remains unallocated by billing currency. +// Scope Notes: +// Heuristic-based; the canonical 'allocated/unallocated' classification depends on each +// organization's allocation rules. Parameterize allocationEvidenceTagKeys to match local +// convention. ResourceId is intentionally not used because the spec did not define whether +// non-resource charges should be excluded from the KPI denominator. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned (SEM0019 fixed with real literal defaults) +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let allocationEvidenceTagKeys = dynamic(['cost-center','team','owner','application','product']); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| extend tmp_TagKeys = coalesce(bag_keys(Tags), dynamic([])) +| extend tmp_HasAllocationEvidenceTag = array_length(set_intersect(tmp_TagKeys, allocationEvidenceTagKeys)) > 0 +| extend tmp_IsUnallocated = isempty(x_CostAllocationRuleName) and isempty(x_CostCenter) and not(tmp_HasAllocationEvidenceTag) +| summarize + UnallocatedEffectiveCost = todouble(sum(iff(tmp_IsUnallocated, EffectiveCost, real(0)))), + TotalEffectiveCost = todouble(sum(EffectiveCost)) + by BillingCurrency +| extend UnallocatedPercent = todouble(iff(TotalEffectiveCost == 0.0, 0.0, UnallocatedEffectiveCost / TotalEffectiveCost * 100.0)) +| project BillingCurrency, UnallocatedEffectiveCost, TotalEffectiveCost, UnallocatedPercent diff --git a/src/queries/catalog/percentage-untagged-costs.kql b/src/queries/catalog/percentage-untagged-costs.kql new file mode 100644 index 000000000..d031791bd --- /dev/null +++ b/src/queries/catalog/percentage-untagged-costs.kql @@ -0,0 +1,29 @@ +// ============================================================================ +// Query: Percentage of Untagged Costs +// Description: +// Calculates the percentage of EffectiveCost attributable to CSP resources with no tags. +// Results are grouped by BillingCurrency and exclude commitment purchases to avoid double-counting amortized costs. +// KPI: Percentage of Costs Associated with Untagged CSP Cloud Resources +// Formula: (Total Effective Costs Associated with Untagged CSP Cloud Resources During a Period of Time / Total CSP Effective Cost During a Period of Time) x 100 +// Author: FinOps toolkit +// Parameters: +// startDate: datetime = startofmonth(ago(30d)) +// endDate: datetime = startofmonth(now()) +// Output: +// Each row represents one BillingCurrency and returns UntaggedEffectiveCost, TotalEffectiveCost, and UntaggedPercent for the selected period. +// Usage: +// Use this query to measure how much effective cloud spend is associated with untagged CSP resources. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +Costs() +| where ChargePeriodStart >= startDate and ChargePeriodStart < endDate +| where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) +| extend isUntagged = isnull(Tags) or array_length(bag_keys(Tags)) == 0 +| summarize + UntaggedEffectiveCost = todouble(sumif(EffectiveCost, isUntagged)), + TotalEffectiveCost = todouble(sum(EffectiveCost)) + by BillingCurrency +| extend UntaggedPercent = todouble(iff(TotalEffectiveCost == 0.0, 0.0, UntaggedEffectiveCost / TotalEffectiveCost * 100.0)) +| project BillingCurrency, UntaggedEffectiveCost, TotalEffectiveCost, UntaggedPercent diff --git a/src/queries/catalog/savings-summary-report.kql b/src/queries/catalog/savings-summary-report.kql index 8d75cbfc3..6634a30a3 100644 --- a/src/queries/catalog/savings-summary-report.kql +++ b/src/queries/catalog/savings-summary-report.kql @@ -18,9 +18,9 @@ let endDate = startofmonth(now()); Costs() | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate | where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) // x_AmortizationCategory != 'Principal' -| extend x_NegotiatedDiscountSavings = iff(ListCost < ContractedCost, decimal(0), ListCost - ContractedCost) -| extend x_CommitmentDiscountSavings = iff(ContractedCost < EffectiveCost, decimal(0), ContractedCost - EffectiveCost) -| extend x_TotalSavings = iff(ListCost < EffectiveCost, decimal(0), ListCost - EffectiveCost) +| extend x_NegotiatedDiscountSavings = iff(ListCost < ContractedCost, real(0), ListCost - ContractedCost) +| extend x_CommitmentDiscountSavings = iff(ContractedCost < EffectiveCost, real(0), ContractedCost - EffectiveCost) +| extend x_TotalSavings = iff(ListCost < EffectiveCost, real(0), ListCost - EffectiveCost) | summarize ListCost = todouble(sum(ListCost)), EffectiveCost = todouble(sum(EffectiveCost)), @@ -28,4 +28,4 @@ Costs() x_CommitmentDiscountSavings = todouble(sum(x_CommitmentDiscountSavings)), x_TotalSavings = todouble(sum(x_TotalSavings)) by BillingCurrency -| extend x_EffectiveSavingsRate = iff(ListCost == 0, 0.0, x_TotalSavings / ListCost * 100.0) \ No newline at end of file +| extend x_EffectiveSavingsRate = iff(ListCost == 0, 0.0, x_TotalSavings / ListCost * 100.0) diff --git a/src/queries/catalog/storage-tier-distribution.kql b/src/queries/catalog/storage-tier-distribution.kql new file mode 100644 index 000000000..5ec2dc7ff --- /dev/null +++ b/src/queries/catalog/storage-tier-distribution.kql @@ -0,0 +1,58 @@ +// ============================================================================ +// Query: Storage Tier Distribution +// Description: +// Summarizes storage usage by access-tier bucket and shows the share of spend and normalized GB-months +// on Frequent, Infrequent, and Unclassified storage tiers for each billing currency. +// KPI: Percent Storage on Frequent Access Tier +// Formula: Number of GB in Standard (or 'frequently accessed' tiers) vs. total GBs stored +// Author: FinOps toolkit +// Parameters: +// startDate: datetime = startofmonth(ago(30d)) +// endDate: datetime = startofmonth(now()) +// Output: +// Each row represents one BillingCurrency and TierBucket pair with EffectiveCost, normalized GBMonths, +// CostPercent, and GBPercent for that bucket within the currency. +// Usage: +// Use this query to measure how much storage cost and GB-month usage remains on frequently accessed tiers. +// Scope Notes: +// Tier classification uses x_SkuTier with fallback to x_SkuMeterSubcategory keyword matching. +// ConsumedQuantity for storage typically represents GB-months over the billed period (time-weighted), not raw GB at a point in time. +// Unclassified rows are intentionally preserved so users can extend the tier-matching logic for uncovered SKUs. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 2 rows returned (Frequent + other tiers) +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let bucketedCosts = + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where ServiceCategory == 'Storage' and ChargeCategory == 'Usage' + | extend TierBucket = case( + x_SkuTier in ('Hot', 'Standard', 'Premium'), 'Frequent', + x_SkuTier in ('Cool', 'Cold', 'Archive'), 'Infrequent', + x_SkuMeterSubcategory has_any ('Hot LRS', 'Hot ZRS', 'Hot GRS', 'Standard', 'Premium', 'Frequent'), 'Frequent', + x_SkuMeterSubcategory has_any ('Cool', 'Cold', 'Archive'), 'Infrequent', + 'Unclassified') + | extend GBMonths = todouble(case( + ConsumedUnit contains 'PB', todouble(ConsumedQuantity) * 1048576.0, + ConsumedUnit contains 'TB', todouble(ConsumedQuantity) * 1024.0, + ConsumedUnit contains 'MB', todouble(ConsumedQuantity) / 1024.0, + todouble(ConsumedQuantity))); +let totalsByCurrency = + bucketedCosts + | summarize + TotalEffectiveCost = sum(EffectiveCost), + TotalGBMonths = sum(GBMonths) + by BillingCurrency; +bucketedCosts +| summarize + EffectiveCost = todouble(sum(EffectiveCost)), + GBMonths = todouble(sum(GBMonths)) + by BillingCurrency, TierBucket +| join kind=inner totalsByCurrency on BillingCurrency +| project + BillingCurrency, + TierBucket, + EffectiveCost, + GBMonths, + CostPercent = todouble(iff(todouble(TotalEffectiveCost) == 0.0, 0.0, EffectiveCost / todouble(TotalEffectiveCost) * 100.0)), + GBPercent = todouble(iff(TotalGBMonths == 0.0, 0.0, GBMonths / TotalGBMonths * 100.0)) diff --git a/src/queries/catalog/tagging-policy-compliance.kql b/src/queries/catalog/tagging-policy-compliance.kql new file mode 100644 index 000000000..1156932d2 --- /dev/null +++ b/src/queries/catalog/tagging-policy-compliance.kql @@ -0,0 +1,56 @@ +// ============================================================================ +// Query: Tagging Policy Compliance +// Description: +// Computes the FinOps Foundation KPI for Tagging Policy Compliance — +// the percentage of effective cloud costs associated with resources that +// satisfy all required tag keys with non-empty values. +// KPI: Percentage of CSP Cloud Costs that are Tagging Policy Compliant +// Formula: (Total Effective Costs Associated with Tagging Policy Compliant CSP Cloud Resources During a Period of Time +// / Total CSP Cloud Effective Costs During a Period of Time) x 100 +// Author: FinOps toolkit +// Parameters: +// startDate: datetime; Start date for the reporting period (default: startofmonth(ago(30d))) +// endDate: datetime; End date for the reporting period (default: startofmonth(now())) +// requiredTagKeys: dynamic; Array of tag key strings that must all be present and non-empty +// (default: dynamic(['cost-center', 'environment', 'owner'])) +// Output: +// BillingCurrency: string; Currency for the billing period; one row per distinct currency. +// CompliantEffectiveCost: double; Sum of effective cost for rows where all required tag keys are present with non-empty values. +// TotalEffectiveCost: double; Sum of effective cost for all rows in scope, excluding commitment purchases. +// CompliancePercent: double; Percentage of total effective cost that is tagging-policy compliant. +// Usage: +// Use to measure and track tag policy adherence across cloud spend over time. +// Scope Notes: +// Policy is defined by the requiredTagKeys parameter. Caller is responsible for +// keeping it in sync with their organization's tag policy. Resources with null Tags +// or missing/empty required-key values are treated as non-compliant. +// Last Tested: 2026-05-28 against a FinOps Hub ADX cluster (1,366,763 cost rows in 2026-04 window). UAT result: PASS — 1 row returned in 3.04 seconds after distinct-tag optimization +// ========================================================================= +let startDate = startofmonth(ago(30d)); +let endDate = startofmonth(now()); +let requiredTagKeys = dynamic(['cost-center', 'environment', 'owner']); +let base = materialize( + Costs() + | where ChargePeriodStart >= startDate and ChargePeriodStart < endDate + | where not(ChargeCategory == 'Purchase' and isnotempty(CommitmentDiscountCategory)) + | project BillingCurrency, EffectiveCost = todouble(EffectiveCost), Tags, tmp_TagsString = tostring(Tags) +); +let tag_compliance = materialize( + base + | summarize Tags = take_any(Tags) by tmp_TagsString + | extend tmp_TagKeys = coalesce(bag_keys(Tags), dynamic([])) + | extend tmp_AllKeysPresent = array_length(set_intersect(tmp_TagKeys, requiredTagKeys)) == array_length(requiredTagKeys) + | mv-apply k = requiredTagKeys to typeof(string) on ( + summarize tmp_AnyValueEmpty = countif(isempty(tostring(Tags[k]))) > 0 + ) + | extend tmp_IsCompliant = tmp_AllKeysPresent and not(tmp_AnyValueEmpty) + | project tmp_TagsString, tmp_IsCompliant +); +base +| join kind=leftouter tag_compliance on tmp_TagsString +| summarize + CompliantEffectiveCost = todouble(sumif(EffectiveCost, tmp_IsCompliant)), + TotalEffectiveCost = todouble(sum(EffectiveCost)) + by BillingCurrency +| extend CompliancePercent = iff(TotalEffectiveCost == 0, 0.0, CompliantEffectiveCost / TotalEffectiveCost * 100.0) +| project BillingCurrency, CompliantEffectiveCost, TotalEffectiveCost, CompliancePercent diff --git a/src/queries/finops-hub-database-guide.md b/src/queries/finops-hub-database-guide.md index a04e8efb1..1ea575922 100644 --- a/src/queries/finops-hub-database-guide.md +++ b/src/queries/finops-hub-database-guide.md @@ -91,13 +91,13 @@ The FinOps hubs database is designed to support advanced cost and usage analytic ## Query Best Practices -- **Start with the [`costs-enriched-base`](./catalog/costs-enriched-base.kql) query:** - Use this query as your base for any cost or usage analytics. It provides the full enrichment and savings logic for all cost columns and is the recommended foundation for custom analytics and reporting. +- **Prefer scenario-specific aggregate queries first:** + Use the query catalog to find the narrowest query that answers the question. Use [`costs-enriched-base`](./catalog/costs-enriched-base.kql) when you need scoped row-level enrichment or repeated drill-downs. -- **Use KQL (Kusto Query Language):** +- **Use KQL (Kusto Query Language):** All queries should be written in KQL for compatibility with Azure Data Explorer. -- **Leverage Enrichment Columns:** +- **Leverage Enrichment Columns:** Columns prefixed with `x_` provide additional context and enrichment for FinOps analysis. --- @@ -192,8 +192,8 @@ Recommendations() x_EffectiveCostBefore, x_LookbackPeriodLabel = replace_regex(tostring(x_RecommendationDetails.LookbackPeriodDuration), 'P([0-9]+)D', @'\1 days'), x_RecommendationDate, - x_RecommendedQuantity = todecimal(x_RecommendationDetails.RecommendedQuantity), - x_RecommendedQuantityNormalized = todecimal(x_RecommendationDetails.RecommendedQuantityNormalized), + x_RecommendedQuantity = toreal(x_RecommendationDetails.RecommendedQuantity), + x_RecommendedQuantityNormalized = toreal(x_RecommendationDetails.RecommendedQuantityNormalized), x_SkuMeterId, x_SkuTerm, x_SkuTermLabel = case(x_SkuTerm < 12, strcat(x_SkuTerm, ' month', iff(x_SkuTerm != 1, 's', '')), strcat(x_SkuTerm / 12, ' year', iff(x_SkuTerm != 12, 's', ''))) @@ -235,10 +235,10 @@ let numberOfMonths = 1; let base = Costs() | where ChargePeriodStart >= monthsago(numberOfMonths) | extend x_SkuCoreCount = toint(coalesce(x_SkuDetails.VCPUs, x_SkuDetails.vCores, '')) -| extend x_ConsumedCoreHours = iff(isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, todecimal('')); -let total = base | summarize Total=todecimal(sum(x_ConsumedCoreHours)); +| extend x_ConsumedCoreHours = iff(isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, toreal('')); +let total = base | summarize Total=toreal(sum(x_ConsumedCoreHours)); base -| summarize TotalConsumedCoreHours = todecimal(sum(x_ConsumedCoreHours)) by CommitmentDiscountType +| summarize TotalConsumedCoreHours = toreal(sum(x_ConsumedCoreHours)) by CommitmentDiscountType | extend CommitmentDiscountType = iff(isempty(CommitmentDiscountType), 'On Demand', CommitmentDiscountType) | extend PercentOfTotal = 100.0 * TotalConsumedCoreHours / toscalar(total) | project CommitmentDiscountType, TotalConsumedCoreHours=todouble(TotalConsumedCoreHours), PercentOfTotal=todouble(PercentOfTotal) @@ -269,7 +269,7 @@ Costs() | extend x_SkuUsageType = tostring(x_SkuDetails.UsageType) | extend x_SkuImageType = tostring(x_SkuDetails.ImageType) | extend x_SkuType = tostring(x_SkuDetails.ServiceType) -| extend x_ConsumedCoreHours = iff(isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, todecimal('')) +| extend x_ConsumedCoreHours = iff(isnotempty(x_SkuCoreCount), x_SkuCoreCount * ConsumedQuantity, toreal('')) | extend x_SkuLicenseStatus = case( ChargeCategory != 'Usage', '', (x_SkuMeterCategory in ('Virtual Machines', 'Virtual Machine Licenses') and x_SkuMeterSubcategory contains 'Windows') or tmp_SQLAHB == 'false', 'Not Enabled', @@ -295,13 +295,13 @@ Costs() // | extend x_CommitmentDiscountKey = iff(tmp_IsVMUsage and isnotempty(x_SkuDetails.ServiceType), strcat(x_SkuDetails.ServiceType, x_SkuMeterId), '') | extend x_CommitmentDiscountUtilizationPotential = case( - ChargeCategory == 'Purchase', decimal(0), + ChargeCategory == 'Purchase', real(0), ProviderName == 'Microsoft' and isnotempty(CommitmentDiscountCategory), EffectiveCost, CommitmentDiscountCategory == 'Usage', ConsumedQuantity, CommitmentDiscountCategory == 'Spend', EffectiveCost, - decimal(0) + real(0) ) -| extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, decimal(0)) +| extend x_CommitmentDiscountUtilizationAmount = iff(CommitmentDiscountStatus == 'Used', x_CommitmentDiscountUtilizationPotential, real(0)) | extend x_SkuTermLabel = case(isempty(x_SkuTerm) or x_SkuTerm <= 0, '', x_SkuTerm < 12, strcat(x_SkuTerm, ' month', iff(x_SkuTerm != 1, 's', '')), strcat(x_SkuTerm / 12, ' year', iff(x_SkuTerm != 12, 's', ''))) // // CSP partners @@ -314,12 +314,12 @@ Costs() isnotempty(CommitmentDiscountCategory), 'Amortized Charge', '' ) -| extend x_CommitmentDiscountSavings = iff(ContractedCost == 0, decimal(0), ContractedCost - EffectiveCost) -| extend x_NegotiatedDiscountSavings = iff(ListCost == 0, decimal(0), ListCost - ContractedCost) -| extend x_TotalSavings = iff(ListCost == 0, decimal(0), ListCost - EffectiveCost) -| extend x_CommitmentDiscountPercent = iff(ContractedUnitPrice == 0, decimal(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice) -| extend x_NegotiatedDiscountPercent = iff(ListUnitPrice == 0, decimal(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice) -| extend x_TotalDiscountPercent = iff(ListUnitPrice == 0, decimal(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice) +| extend x_CommitmentDiscountSavings = iff(ContractedCost == 0, real(0), ContractedCost - EffectiveCost) +| extend x_NegotiatedDiscountSavings = iff(ListCost == 0, real(0), ListCost - ContractedCost) +| extend x_TotalSavings = iff(ListCost == 0, real(0), ListCost - EffectiveCost) +| extend x_CommitmentDiscountPercent = iff(ContractedUnitPrice == 0, real(0), (ContractedUnitPrice - x_EffectiveUnitPrice) / ContractedUnitPrice) +| extend x_NegotiatedDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - ContractedUnitPrice) / ListUnitPrice) +| extend x_TotalDiscountPercent = iff(ListUnitPrice == 0, real(0), (ListUnitPrice - x_EffectiveUnitPrice) / ListUnitPrice) // // Toolkit | extend x_ToolkitTool = tostring(Tags['ftk-tool']) @@ -359,9 +359,12 @@ Costs() ## Table reference -> **Note:** +> **Note:** > All columns prefixed with `x_` are toolkit enrichment columns, providing additional context for FinOps analysis. +> **Numeric column types — live Hub behavior:** +> Columns documented below as `real` are exposed by the live Hub as `real` (a 64-bit floating-point type, equivalent to KQL `double` / .NET `System.Double`). Earlier versions of this guide listed numeric columns as `decimal`, but the deployed Hub schemas across `Costs()`, `Prices()`, and `Recommendations()` use `real`. When writing literal numeric defaults in KQL queries (for example the `false`-branch of an `iff(...)` aggregate), use `real(0)` — not `decimal(0)` — to avoid `SEM0019` arithmetic type-mismatch errors. + Below are the column definitions for the main analytic tables in the FinOps hubs database. These definitions are based on Microsoft Learn, FinOps best practices, and common cloud cost management terminology. --- @@ -373,7 +376,7 @@ The following table lists the columns produced in the `All available columns` qu | Column Name | Data Type | Description | | ---------------------------------------- | --------- | --------------------------------------------------------------------------------- | | AvailabilityZone | string | Availability zone of the resource, if applicable. | -| BilledCost | decimal | The cost billed for the resource or usage. | +| BilledCost | real | The cost billed for the resource or usage. | | BillingAccountId | string | Unique identifier for the billing account. | | BillingAccountName | string | Name of the billing account. | | BillingAccountType | string | Type of billing account (e.g., EA, MCA). | @@ -391,16 +394,16 @@ The following table lists the columns produced in the `All available columns` qu | CommitmentDiscountName | string | Name of the commitment discount. | | CommitmentDiscountStatus | string | Status of the commitment discount (e.g., Used, Unused). | | CommitmentDiscountType | string | The specific type of discount (e.g., RI, SP). | -| ConsumedQuantity | decimal | Amount of resource usage consumed. | +| ConsumedQuantity | real | Amount of resource usage consumed. | | ConsumedUnit | string | Unit of measure for consumed quantity. | -| ContractedCost | decimal | Negotiated cost for the resource or usage. | -| ContractedUnitPrice | decimal | Negotiated unit price for the resource. | -| EffectiveCost | decimal | Actual cost after all discounts and credits. | +| ContractedCost | real | Negotiated cost for the resource or usage. | +| ContractedUnitPrice | real | Negotiated unit price for the resource. | +| EffectiveCost | real | Actual cost after all discounts and credits. | | InvoiceIssuerName | string | Name of the invoice issuer. | -| ListCost | decimal | List (retail) cost for the resource or usage. | -| ListUnitPrice | decimal | List (retail) unit price for the resource. | +| ListCost | real | List (retail) cost for the resource or usage. | +| ListUnitPrice | real | List (retail) unit price for the resource. | | PricingCategory | string | Category of pricing (e.g., Standard, Spot). | -| PricingQuantity | decimal | Quantity used for pricing. | +| PricingQuantity | real | Quantity used for pricing. | | PricingUnit | string | Unit of measure for pricing. | | ProviderName | string | Name of the cloud provider. | | PublisherName | string | Name of the publisher. | @@ -420,40 +423,40 @@ The following table lists the columns produced in the `All available columns` qu | x_AccountId | string | Enriched account identifier. | | x_AccountName | string | Enriched account name. | | x_AccountOwnerId | string | Owner ID for the account. | -| x_BilledCostInUsd | decimal | Billed cost converted to USD. | -| x_BilledUnitPrice | decimal | Billed unit price. | +| x_BilledCostInUsd | real | Billed cost converted to USD. | +| x_BilledUnitPrice | real | Billed unit price. | | x_BillingAccountAgreement | string | Billing agreement reference. | | x_BillingAccountId | string | Enriched billing account ID. | | x_BillingAccountName | string | Enriched billing account name. | -| x_BillingExchangeRate | decimal | Exchange rate used for billing. | +| x_BillingExchangeRate | real | Exchange rate used for billing. | | x_BillingExchangeRateDate | datetime | Date of the exchange rate. | | x_BillingProfileId | string | Billing profile identifier. | | x_BillingProfileName | string | Name of the billing profile. | | x_ChargeId | string | Unique identifier for the charge. | -| x_ContractedCostInUsd | decimal | Contracted cost converted to USD. | +| x_ContractedCostInUsd | real | Contracted cost converted to USD. | | x_CostAllocationRuleName | string | Name of the cost allocation rule. | | x_CostCategories | dynamic | Cost categories as a dynamic object. | | x_CostCenter | string | Cost center for the transaction. | | x_Credits | dynamic | Credits applied as a dynamic object. | | x_CostType | string | Type of cost (e.g., Amortized, Principal). | -| x_CurrencyConversionRate | decimal | Currency conversion rate used. | +| x_CurrencyConversionRate | real | Currency conversion rate used. | | x_CustomerId | string | Customer identifier. | | x_CustomerName | string | Name of the customer. | | x_Discount | dynamic | Discount details as a dynamic object. | -| x_EffectiveCostInUsd | decimal | Effective cost converted to USD. | -| x_EffectiveUnitPrice | decimal | Final unit price after all discounts. | +| x_EffectiveCostInUsd | real | Effective cost converted to USD. | +| x_EffectiveUnitPrice | real | Final unit price after all discounts. | | x_ExportTime | datetime | Time the record was exported. | | x_IngestionTime | datetime | Timestamp when the record was ingested. | | x_InvoiceId | string | Invoice identifier. | | x_InvoiceIssuerId | string | Invoice issuer identifier. | | x_InvoiceSectionId | string | Invoice section identifier. | | x_InvoiceSectionName | string | Invoice section name. | -| x_ListCostInUsd | decimal | List cost converted to USD. | +| x_ListCostInUsd | real | List cost converted to USD. | | x_Location | string | Location of the resource. | | x_Operation | string | Operation performed. | | x_PartnerCreditApplied | string | Whether partner credit was applied. | | x_PartnerCreditRate | string | Partner credit rate. | -| x_PricingBlockSize | decimal | Block size for pricing. | +| x_PricingBlockSize | real | Block size for pricing. | | x_PricingCurrency | string | Currency for pricing. | | x_PricingSubcategory | string | Subcategory for pricing. | | x_PricingUnitDescription | string | Description of the pricing unit. | @@ -495,23 +498,23 @@ The following table lists the columns produced in the `All available columns` qu | x_SkuUsageType | string | Usage type for the SKU. | | x_SkuImageType | string | Image type for the SKU. | | x_SkuType | string | Service type for the SKU. | -| x_ConsumedCoreHours | decimal | Total core hours consumed. | +| x_ConsumedCoreHours | real | Total core hours consumed. | | x_SkuLicenseStatus | string | License status for the SKU. | | x_SkuLicenseType | string | License type for the SKU. | | x_SkuLicenseQuantity | long | License quantity for the SKU. | | x_SkuLicenseUnit | string | License unit for the SKU. | | x_SkuLicenseUnusedQuantity | long | Unused license quantity for the SKU. | | x_CommitmentDiscountKey | string | Key for commitment discount utilization. | -| x_CommitmentDiscountUtilizationPotential | decimal | Potential utilization for commitment discount. | -| x_CommitmentDiscountUtilizationAmount | decimal | Actual utilization amount for commitment discount. | +| x_CommitmentDiscountUtilizationPotential | real | Potential utilization for commitment discount. | +| x_CommitmentDiscountUtilizationAmount | real | Actual utilization amount for commitment discount. | | x_SkuTermLabel | string | Human-readable label for SKU term. | | x_AmortizationCategory | string | Amortization category (e.g., Principal, Amortized Charge). | -| x_CommitmentDiscountSavings | decimal | Realized savings from commitment discounts (actual savings applied to your bill). | -| x_NegotiatedDiscountSavings | decimal | Realized savings from negotiated discounts (actual savings applied to your bill). | -| x_TotalSavings | decimal | Realized total savings (negotiated + commitment, as actually applied). | -| x_CommitmentDiscountPercent | decimal | Percent savings from commitment discount. | -| x_NegotiatedDiscountPercent | decimal | Percent savings from negotiated discount. | -| x_TotalDiscountPercent | decimal | Total percent savings. | +| x_CommitmentDiscountSavings | real | Realized savings from commitment discounts (actual savings applied to your bill). | +| x_NegotiatedDiscountSavings | real | Realized savings from negotiated discounts (actual savings applied to your bill). | +| x_TotalSavings | real | Realized total savings (negotiated + commitment, as actually applied). | +| x_CommitmentDiscountPercent | real | Percent savings from commitment discount. | +| x_NegotiatedDiscountPercent | real | Percent savings from negotiated discount. | +| x_TotalDiscountPercent | real | Total percent savings. | | x_ToolkitTool | string | Toolkit tool name. | | x_ToolkitVersion | string | Toolkit version. | | x_ResourceParentId | string | Resource parent identifier. | @@ -538,34 +541,34 @@ The following table lists the columns produced in the `All available columns` qu | ChargeCategory | string | Category of the charge (e.g., Usage, Purchase). | | CommitmentDiscountCategory | string | Type of commitment discount. | | CommitmentDiscountType | string | Specific type of discount. | -| ContractedUnitPrice | decimal | Negotiated unit price for the resource. | -| ListUnitPrice | decimal | List (retail) unit price for the resource. | +| ContractedUnitPrice | real | Negotiated unit price for the resource. | +| ListUnitPrice | real | List (retail) unit price for the resource. | | PricingCategory | string | Category of pricing (e.g., Standard, Spot). | | PricingUnit | string | Unit of measure for pricing (e.g., hours, GB). | | SkuId | string | Unique identifier for the SKU. | | SkuPriceId | string | Unique identifier for the SKU price. | | SkuPriceIdv2 | string | Alternate identifier for the SKU price. | -| x_BaseUnitPrice | decimal | Base unit price before discounts. | +| x_BaseUnitPrice | real | Base unit price before discounts. | | x_BillingAccountAgreement | string | Billing agreement reference. | | x_BillingAccountId | string | Enriched billing account ID. | | x_BillingProfileId | string | Billing profile identifier. | | x_CommitmentDiscountSpendEligibility | string | Eligibility for spend-based discounts. | | x_CommitmentDiscountUsageEligibility | string | Eligibility for usage-based discounts. | -| x_ContractedUnitPriceDiscount | decimal | Discount amount from contracted price. | -| x_ContractedUnitPriceDiscountPercent | decimal | Discount percent from contracted price. | +| x_ContractedUnitPriceDiscount | real | Discount amount from contracted price. | +| x_ContractedUnitPriceDiscountPercent | real | Discount percent from contracted price. | | x_EffectivePeriodEnd | datetime | End of the effective pricing period. | | x_EffectivePeriodStart | datetime | Start of the effective pricing period. | -| x_EffectiveUnitPrice | decimal | Final unit price after all discounts. | -| x_EffectiveUnitPriceDiscount | decimal | Discount amount from effective price. | -| x_EffectiveUnitPriceDiscountPercent | decimal | Discount percent from effective price. | +| x_EffectiveUnitPrice | real | Final unit price after all discounts. | +| x_EffectiveUnitPriceDiscount | real | Discount amount from effective price. | +| x_EffectiveUnitPriceDiscountPercent | real | Discount percent from effective price. | | x_IngestionTime | datetime | Timestamp when the record was ingested. | -| x_PricingBlockSize | decimal | Block size for pricing (e.g., 1000 units). | +| x_PricingBlockSize | real | Block size for pricing (e.g., 1000 units). | | x_PricingCurrency | string | Currency for pricing. | | x_PricingSubcategory | string | Subcategory for pricing. | | x_PricingUnitDescription | string | Description of the pricing unit. | | x_SkuDescription | string | Description of the SKU. | | x_SkuId | string | Enriched SKU ID. | -| x_SkuIncludedQuantity | decimal | Quantity included at no extra cost. | +| x_SkuIncludedQuantity | real | Quantity included at no extra cost. | | x_SkuMeterCategory | string | Meter category for the SKU. | | x_SkuMeterId | string | Meter ID for the SKU. | | x_SkuMeterName | string | Meter name for the SKU. | @@ -578,32 +581,43 @@ The following table lists the columns produced in the `All available columns` qu | x_SkuOfferId | string | Offer ID for the SKU. | | x_SkuPartNumber | string | Part number for the SKU. | | x_SkuTerm | int | Term length for the SKU (months). | -| x_SkuTier | decimal | Tier for the SKU (e.g., Standard, Premium). | +| x_SkuTier | real | Tier for the SKU (e.g., Standard, Premium). | | x_SourceName | string | Name of the data source. | | x_SourceProvider | string | Provider of the data source. | | x_SourceType | string | Type of data source. | | x_SourceVersion | string | Version of the data source. | -| x_TotalUnitPriceDiscount | decimal | Total discount amount. | -| x_TotalUnitPriceDiscountPercent | decimal | Total discount percent. | +| x_TotalUnitPriceDiscount | real | Total discount amount. | +| x_TotalUnitPriceDiscountPercent | real | Total discount percent. | --- ### Recommendations() -| Column Name | Data Type | Description | -| ----------------------- | --------- | ------------------------------------------------- | -| ProviderName | string | Name of the cloud provider. | -| SubAccountId | string | Identifier for the sub-account or subscription. | -| x_IngestionTime | datetime | Timestamp when the record was ingested. | -| x_EffectiveCostAfter | decimal | Projected cost after applying the recommendation. | -| x_EffectiveCostBefore | decimal | Cost before applying the recommendation. | -| x_EffectiveCostSavings | decimal | Estimated cost savings from the recommendation. | -| x_RecommendationDate | datetime | Date the recommendation was generated. | -| x_RecommendationDetails | dynamic | Details of the recommendation (dynamic object). | -| x_SourceName | string | Name of the data source. | -| x_SourceProvider | string | Provider of the data source. | -| x_SourceType | string | Type of data source. | -| x_SourceVersion | string | Version of the data source. | +> **Sparsely-populated columns:** +> The `Recommendations()` function is sourced from upstream optimization recommenders (for example, Azure Advisor reservation recommendations). Many string identifier columns below (`ResourceId`, `ResourceName`, `SubAccountId`, `SubAccountName`, `x_RecommendationId`, `x_RecommendationDescription`, `x_ResourceGroupName`) may be empty strings in practice if the source recommender does not surface them. Verify population with `Recommendations() | summarize count() by isnotempty()` before relying on a column as a join key or filter. + +| Column Name | Data Type | Description | +| ---------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| ProviderName | string | Name of the cloud provider. | +| ResourceId | string | Unique identifier for the resource the recommendation applies to. | +| ResourceName | string | Display name of the resource. | +| ResourceType | string | Type of the resource (e.g., `virtualmachines`). | +| SubAccountId | string | Identifier for the sub-account or subscription. | +| SubAccountName | string | Display name of the sub-account or subscription. | +| x_EffectiveCostAfter | real | Projected cost after applying the recommendation. | +| x_EffectiveCostBefore | real | Cost before applying the recommendation. | +| x_EffectiveCostSavings | real | Estimated cost savings from the recommendation. | +| x_IngestionTime | datetime | Timestamp when the record was ingested. Always populated. | +| x_RecommendationCategory | string | Category of the recommendation (e.g., `ReservationRecommendations`, `SavingsPlanRecommendations`). | +| x_RecommendationDate | datetime | Date the recommendation was generated by the source recommender. **Commonly null in live Hubs** — the upstream recommender may not stamp this field. Do not filter `Recommendations()` by `x_RecommendationDate` with a date-window predicate (e.g., `where x_RecommendationDate >= startDate`) — KQL `null >= datetime(…)` evaluates false, silently dropping every row. Use `x_IngestionTime` as the time anchor. | +| x_RecommendationDescription | string | Human-readable description of the recommendation. | +| x_RecommendationDetails | dynamic | Structured details of the recommendation (dynamic object — shape varies by category). | +| x_RecommendationId | string | Unique identifier for the recommendation. | +| x_ResourceGroupName | string | Resource group containing the resource (Azure only). | +| x_SourceName | string | Name of the data source. | +| x_SourceProvider | string | Provider of the data source. | +| x_SourceType | string | Type of data source. | +| x_SourceVersion | string | Version of the data source. | --- @@ -675,10 +689,11 @@ The following table lists the columns produced in the `All available columns` qu ## Change log -| Date | Version | Author | Notes | -| ---------- | ------- | ------------------- | ------------------------------------- | -| 2025-05-16 | 1.0 | FinOps Toolkit Team | Initial documentation | -| 2025-05-16 | 1.1 | FinOps Toolkit Team | Expanded schema, glossary, references | +| Date | Version | Author | Notes | +| ---------- | ------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| 2025-05-16 | 1.0 | FinOps Toolkit Team | Initial documentation | +| 2025-05-16 | 1.1 | FinOps Toolkit Team | Expanded schema, glossary, references | +| 2026-05-28 | 1.2 | Sprint 3000 UAT | Live-Hub schema audit: `Costs()`, `Prices()`, `Recommendations()` numeric columns retyped from `decimal` to `real` to match deployed Hub schema (cause of SEM0019 errors). `Recommendations()` table expanded from 12 to 20 columns to add the 8 columns present in the live schema. `x_RecommendationDate` documented as commonly-null in live Hubs (root cause of T-3000.13). | ---