Skip to content

Fix duplicate guest trace batches#1619

Open
ludfjig wants to merge 2 commits into
hyperlight-dev:mainfrom
ludfjig:fix-dup-guest-traces
Open

Fix duplicate guest trace batches#1619
ludfjig wants to merge 2 commits into
hyperlight-dev:mainfrom
ludfjig:fix-dup-guest-traces

Conversation

@ludfjig

@ludfjig ludfjig commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

The host stamped the TraceBatch magic in r8 on every OUT and polled r8 on every VM exit. r8 is sticky, so the host re-read the same batch on later exits and emitted duplicate spans.

Deliver batches on a dedicated TraceBatch port. The host reads guest registers only on that exit, so each batch is read once. Aborts flush the buffer over the same port so the final batch still reaches the host.

Closes #1230

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes duplicate guest trace span batches by moving trace-batch delivery to a dedicated TraceBatch OUT port, so the host only reads the trace registers on the specific VM exit that carries a new batch (instead of polling sticky registers on every exit).

Changes:

  • Host: only reads guest registers and parses trace data on VmExit::IoOut where the port is OutBAction::TraceBatch.
  • Guest: trace batches are emitted via a dedicated TraceBatch OUT in the tracing subsystem; abort paths now call hyperlight_guest_tracing::flush() before emitting abort bytes.
  • Cleanup/docs: removes the old “piggyback trace batch on arbitrary OUT” API surface and updates tracing documentation accordingly.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/hyperlight_host/src/sandbox/trace/context.rs Removes the old “check registers for trace data” helper; trace handling is now driven by dedicated exits.
src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs Parses trace batches only on TraceBatch port exits, avoiding re-reading sticky registers on unrelated exits.
src/hyperlight_guest/src/exit.rs Flushes trace batches before deliberate abort sequences so the final buffered batch can be delivered.
src/hyperlight_guest/src/arch/amd64/exit.rs Makes out32 a pure I/O primitive again (no embedded trace-batch side effects).
src/hyperlight_guest_tracing/src/state.rs Sends trace batches to the host via a dedicated TraceBatch OUT and tightens inline-asm options.
src/hyperlight_guest_tracing/src/lib.rs Trims exported API to the remaining supported tracing entrypoints.
docs/hyperlight-metrics-logs-and-traces.md Documents the dedicated TraceBatch exit and new batching/flush behavior (with a couple clarifications needed per review comments).

Comment thread docs/hyperlight-metrics-logs-and-traces.md
Comment thread docs/hyperlight-metrics-logs-and-traces.md Outdated
ludfjig added 2 commits July 2, 2026 18:28
The host stamped the TraceBatch magic in r8 on every OUT and polled r8 on
every VM exit. r8 is sticky, so the host re-read the same batch on later
exits and emitted duplicate spans.

Deliver batches on a dedicated TraceBatch port. The host reads guest
registers only on that exit, so each batch is read once. Aborts flush the
buffer over the same port so the final batch still reaches the host.

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig force-pushed the fix-dup-guest-traces branch from a2fcbca to 9c871f5 Compare July 3, 2026 01:29
@ludfjig ludfjig added the kind/bugfix For PRs that fix bugs label Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bugfix For PRs that fix bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix duplicate guest traces

2 participants