[token-consumption] Daily AIC Consumption Report - 2026-06-12

[github-actions]

Executive Summary

AIC (AI Credits) usage was triaged across all agentic workflows in github/gh-aw for the last 24h using OpenTelemetry spans in both Sentry and Grafana/Tempo.

• Confirmed total AIC: ≈ 778.3 AIC (lower bound — see caveats). This is sourced from Grafana/Tempo only; Sentry AIC was not queryable.
• Top consumer by far: Typist - Go Type Analysis at ≈ 634.5 AIC (≈ 82% of all confirmed usage), driven almost entirely by a single agent invocation worth 624.886 AIC.
• Unique workflows observed: ≥ 50 (span-emitting), but only 5 workflows had confirmed non-zero AIC.
• Sentry AIC queryability: ❌ NOT queryable — gh-aw.aic returns no results for has:, :>0, or :>=0, and renders empty in span results, even though count() and other gh-aw.* fields aggregate normally.
• Grafana AIC queryability: ⚠️ partial — float values are present and indexed, but numeric range filters (> 0) silently miss them due to an int/double type mismatch. Values are recoverable only by exact match.

This is primarily an observability report on a telemetry gap: AIC is being emitted but is not aggregatable in either backend by the normal query path.

Key Metrics

Metric	Value
Events analyzed (Sentry, canonical)	~3,445 spans across top 50 workflows (24h; page/list capped)
Events with AIC data (confirmed numeric, non-zero)	12 spans (Grafana)
Events with AIC data (Sentry)	0 (field not queryable)
Events with AIC data (Grafana)	12 spans (8 distinct values), exact-match only
Total AIC	≈ 778.3 AIC (Grafana-derived; Sentry yields none)
Unique workflows	≥ 50 emitting spans; 5 with confirmed non-zero AIC
Avg AIC/event (AIC-bearing spans, n=8 values)	≈ 97.3 AIC
P95 AIC/event (AIC-bearing spans, n=8 values)	≈ 624.9 AIC (dominated by Typist)

Note: avg/p95 are computed over the 8 confirmed AIC-bearing job spans only. Across all spans the mean is ≈ 0 because the vast majority report gh-aw.aic as integer 0 (the type-mismatch culprit) or omit it.

Top 10 Workflows by AIC Consumption

Only 5 workflows had confirmed non-zero AIC. Each value verified by exact-match TraceQL and cross-checked to a workflow via rootServiceName / Sentry gh-aw.workflow.name.

Workflow	AIC-bearing jobs	Total AIC	Highest single span (job / run)
Typist - Go Type Analysis	2	634.534	624.886 — agent (run 27415555013)
PR Sous Chef	1	71.070	71.070 — run 27416420500
Constraint Solving — Problem of the Day	2	48.919	44.573 — agent (run 27415660496, failed)
Daily Go Function Namer	2	12.026	9.558 — agent (run 27416267062)
Package Specification Enforcer	1	11.740	11.740 — agent (run 27415689181)
Confirmed total		≈ 778.289

Per-workflow AIC breakdown (agent vs detection)

• Typist - Go Type Analysis — agent: 624.886, detection: 9.648 (trace 5155a5afed37cd8edbd9398241c2f0b0)
• PR Sous Chef — 71.07 (trace bf4e119c20b4e5cb8a9ad77af227d68a)
• Constraint Solving — POTD — agent: 44.573, detection: 4.346 (trace b22b19b6a871b47117840f9b558713b5; run concluded failure)
• Daily Go Function Namer — agent: 9.558, detection: 2.468 (trace 2ea0a8315b9a93bcd1e5c86de79f0093)
• Package Specification Enforcer — 11.74 (trace 47af89933b6ec7142d7d20551e110a21)

Grafana AIC Findings

AIC is only partially queryable in Grafana/Tempo. The attribute is indexed and the float values are physically present, but the normal aggregation path (numeric range filters) is broken.

• ✅ tempo_get-attribute-values on span.gh-aw.aic returns float values: 624.886, 71.07, 44.573, 11.74, 9.648, 9.558, 4.346, 2.468.
• ✅ Exact match works: { span."gh-aw.aic" = 624.886 } → trace 5155a5afed..., doubleValue: 624.886, service gh-aw.typist.
• ❌ Range filters fail: { span."gh-aw.aic" > 0 } and { span."gh-aw.aic" > 1 } return zero traces; { span."gh-aw.aic" >= 0 } returns only spans encoded as intValue: 0.

Root cause: spans with AIC = 0 are stored as intValue:0, while non-zero AIC is stored as doubleValue. The mixed int/double typing for one key makes TraceQL numeric range comparisons skip the float-typed spans. actions/setup/js/send_otlp_span.cjs already forces doubleValue via buildDoubleAttr("gh-aw.aic", aiCredits) (line ~2121), but 0.0 is being coerced back to integer 0 somewhere in the OTLP export/Tempo ingest path, recreating the exact mismatch the code comment warns about. Net effect: numeric aggregation/top-N by AIC is impossible in Grafana without per-value exact-match enumeration.

Data Quality and Gaps

• Sentry AIC fully unqueryable. has:gh-aw.aic, gh-aw.aic:>0, gh-aw.aic:>=0 all return no results and the field is empty in span output — yet count() grouped by gh-aw.workflow.name works fine, so this is specific to the AIC attribute's schema/ingest in Sentry EAP, not a general query failure.
• Many token-owning jobs report AIC = 0. Workflows that clearly ran agents (e.g. Issue Monster, Draft PR Cleanup, Daily Documentation Updater, Daily Choice Type Test, GitHub API Consumption Report Agent, several Smoke-* runs) show gh-aw.aic as integer 0. Per the emit logic this likely reflects the firewall proxy writing ai_credits=0 to agent_usage.json or the engine not reporting credits — i.e., an emission gap, not true zero usage. These are reported as unknown AIC, not zero consumption.
• Total is a confirmed lower bound. tempo_get-attribute-values returns distinct values and may be capped; non-aggregatable backends prevent a guaranteed-complete sum.
• Events missing workflow identifier: negligible — gh-aw.workflow.name populated on essentially all spans queried. One sub-gen_ai span lacked gh-aw.run.status (expected — that attribute is conclusion-span-only).
• Companion datasets: scope was AIC on spans; gh-aw.run.status:failure is observable (Constraint Solving — POTD failed while still consuming 48.9 AIC). Errors/logs datasets were not the focus of this AIC triage.
• Assumptions: each distinct float value counted once per job; agent+detection summed per workflow; cross-backend continuity confirmed via matching span IDs (Tempo 885be6af85e67553 ↔ Sentry Typist agent span).

Recommendations

1. Fix the AIC type mismatch at the source. Ensure gh-aw.aic is encoded as doubleValue end-to-end so the int/double split disappears — verify the OTLP exporter/Tempo are not coercing 0.0 → intValue:0. Until then, omit gh-aw.aic when the value is 0 (rather than emitting an integer 0) so the attribute stays single-typed (double) and { span."gh-aw.aic" > 0 } works. This single change makes both top-N and totals queryable in Grafana.
2. Investigate Sentry EAP AIC ingestion. gh-aw.aic is unqueryable in Sentry despite count() working; confirm the numeric attribute is being indexed (field schema / measurement namespace) so sum(gh-aw.aic) and gh-aw.aic:>0 resolve.
3. Close the AIC = 0 emission gap. Trace why agent-owning jobs report ai_credits=0 (firewall proxy zero-write vs engine result event) so credits aren't silently lost — today only 5 of ≥50 workflows surface any AIC.
4. Throttle the dominant consumer. Typist - Go Type Analysis alone is ~82% of confirmed AIC (≈634.5, with one 624.886 agent run). Review its model tier, turn count, and prompt size; a single workflow this dominant is the highest-leverage place to cut spend. Also flag Constraint Solving — POTD, which burned ≈49 AIC on a run that concluded in failure.

References

• Sentry — AIC query returning no results (gap evidence): https://github.sentry.io/explore/traces/?query=gh-aw.aic:%3E0&project=4511347087179777&statsPeriod=24h&table=span
• Sentry — per-workflow span counts (count() aggregation works): https://github.sentry.io/explore/traces/?query=has:gh-aw.workflow.name&project=4511347087179777&statsPeriod=24h&table=span
• Grafana/Tempo — datasource grafanacloud-traces; TraceQL { span."gh-aw.aic" = 624.886 } → top-consumer trace 5155a5afed37cd8edbd9398241c2f0b0 (service gh-aw.typist)
• Top consumer run: https://github.com/github/gh-aw/actions/runs/27415555013

Generated by 📊 Daily AIC Consumption Report (Sentry + Grafana OTel) · 241.9 AIC · ⌖ 31.6 AIC · ⊞ 7.4K · ◷

• expires on Jun 13, 2026, 4:57 AM UTC-08:00