agentHost: Track tool result size

[amunger]

Overview

What

Adds resultSizeInCharacters to the canonical languageModelToolInvoked event for Agent Host tool completions. The measurement is computed as the serialized AHP ToolCallResult length in UTF-16 code units and flows through the existing AgentSideEffects -> AgentHostToolCallTracker -> AgentHostTelemetryReporter path.

This preserves the result-size analysis currently provided locally by agent.tool.responseLength without duplicating that legacy event name or schema. The local emitter is unchanged.

Follow-up work may add tokenizer-aware result cost, model identity, and request-level correlation if those dimensions are needed. They are deliberately not represented by placeholders here.

Why

Tracked in https://github.com/microsoft/vscode-internalbacklog/issues/8247. The local event was observed in local sessions but had no Agent Host counterpart, leaving tool-result-size analysis dark for AH traffic. Character count is an honest close analogue that supports relative size distributions while keeping the unit difference explicit.

---

Data parity

Legend: ✅ = same analytical signal, ≈ = close analogue, ⊗ = AH data omits.

Analytical signal	Local event / source	AH event / source	Parity
Tool identity	agent.tool.responseLength.toolName	languageModelToolInvoked.toolId	✅ after renaming
Successful result population	Event fires only after success	Filter languageModelToolInvoked.result == "success"	✅ after filtering
Relative result size	Tokenized tokenCount	Serialized resultSizeInCharacters	≈ different unit and envelope
Model-normalized prompt cost	Endpoint tokenizer	Not available	⊗ pending SDK support
Model segmentation	model	Not available on this event	⊗ pending plumbing
Request correlation	conversationId, requestId	chatSessionId only	≈ session-level only

---

Row-count and dashboard implications

• Historical rows: agent.tool.responseLength remains success-only and token-based.
• AH rows: languageModelToolInvoked covers successful, failed, and user-cancelled completed tool calls; filter to result == "success" for population parity.
• Query translation: normalize toolName = coalesce(toolName, toolId) and retain an explicit size-unit dimension. Do not coalesce raw token and character counts into one numeric series.
• Privacy: only the numeric serialized length is emitted; tool-result content is not added to telemetry.

Semantic-shift ledger

Signal	Shift	Impact
Size unit	Model tokens become serialized UTF-16 code units	Compare distributions within one unit/population; raw values are not interchangeable
Counted payload	Rendered primitive result becomes the serialized AHP result envelope	AH values include fixed protocol/structured metadata overhead
Cadence	Success-only becomes all completed outcomes	Filter AH rows to successful outcomes for parity

Validation

• scripts/test.bat --run src/vs/platform/agentHost/test/node/agentHostToolCallTelemetry.test.ts — 14 passing
• npm run precommit — passed
• npm run typecheck-client and npm run valid-layers-check were attempted but are currently blocked by unrelated pre-existing Agent Host protocol type errors in copilotAgentSession.ts, copilotSystemNotification.ts, and fixtureUtils.ts.

[Copilot]

Pull request overview

Adds serialized Agent Host tool-result size telemetry to the canonical languageModelToolInvoked event.

Changes:

• Measures ToolCallResult JSON length in UTF-16 code units.
• Propagates the measurement through Agent Host telemetry.
• Tests successful, failed, client, and content-bearing results.

Show a summary per file

File	Description
languageModelToolTelemetry.ts	Defines and classifies the new measurement.
agentHostToolCallTelemetry.test.ts	Verifies emitted result sizes.
agentHostToolCallTracker.ts	Computes serialized result length.
agentHostTelemetryReporter.ts	Emits the measurement.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 0
• Review effort level: Balanced