Skip to content

ProgressEvent has no tool-completion milestone, so a host cannot report a tool's outcome #88

Description

@senamakel

What is missing

ProgressEvent reports that a tool started (ToolCall { run, call, tool }) but never that it finished. There is no success flag, no output, and no duration anywhere in the stream.

A host adapter therefore cannot close out a tool row. In OpenHuman (tinyhumansai/openhuman#5396) this leaves every timeline entry in Running forever and the frontend never learns whether the tool succeeded — AgentProgress::ToolCallCompleted requires success: bool, output_chars, and output, and none of the three is derivable from the coarse stream.

Why the obvious workarounds are worse

  • Infer completion from the next event. Sound as far as it goes — the runtime only proceeds once the tool returns — but it establishes that the tool finished, not how. The adapter would still have to assert an outcome.
  • Default success: true. This is the one to avoid. It marks failed tools as successful in the timeline and in the trace exporter's span output: wrong data rather than missing data, and far harder to notice than a row visibly stuck in Running.

Why this is not the presentation leak the module doc warns against

The module doc is explicit that ProgressEvent should stay at five variants, and that reasoning is right — timeline entries, cost footers and citation chips are host contracts and must not migrate into this enum.

The distinction I would argue for here: "this tool call ended, and it did or did not succeed" is a runtime fact, not a presentation concern. It is the same category as Started and Finished, both of which already exist. Only the runtime knows it; no host can derive it. That is the opposite of a chip or a footer, which the host computes from runtime facts.

The asymmetry is what makes it look like an oversight rather than a decision: Started/Finished bracket a run, ToolCall opens a tool and nothing closes it.

Suggested shape

ToolCallFinished {
    run: RunId,
    call: CallId,
    success: bool,
    /// Raw tool output, or empty when the runtime ran with payload capture off.
    output: String,
},

call correlates with the opening ToolCall. output deliberately carries no truncation or redaction policy — both are host decisions, and OpenHuman's adapter already applies them.

Impact

This is a Phase 4 repointing blocker on the OpenHuman side (see docs/specs/plan-agents.md): OpenHumanProgressSink should not go live driving the real progress bridge until tool completion can be reported truthfully. The gap is documented as a named contract mismatch in that adapter's module header, with the reasoning, so nobody later "fixes" it by defaulting success to true.

Found by review on tinyhumansai/openhuman#5396.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions