Skip to content

Prevent MSRs from leaking across calls to MultiuseSandbox::restore#991

Open
ludfjig wants to merge 1 commit into
hyperlight-dev:mainfrom
ludfjig:reset2
Open

Prevent MSRs from leaking across calls to MultiuseSandbox::restore#991
ludfjig wants to merge 1 commit into
hyperlight-dev:mainfrom
ludfjig:reset2

Conversation

@ludfjig

@ludfjig ludfjig commented Oct 27, 2025

Copy link
Copy Markdown
Contributor

The goal is to prevent guest state to persist across snapshot-restores.

  • Kvm: directly disallows read/writes to msrs using KVM_X86_SET_MSR_FILTER. The guest will trap immediately and error will be returned
  • Hyperv (mshv+whp): The read/writes are allowed (no way to prevent it (intercept exists but doesn't cover all msrs)). Instead restore will reset set msrs using a hardcoded list of MSRs that are known to be writeable. The baseline values of these MSRs are captured on vm-creation. A limitation is that any MSR that is guest-writable and not in this list, has the potential to leak.

Future: When scrub-parition hvcall is ready, we will switch to it, which is guaranteed reset all MSRs. (won't exist for kvm though, which is fine)

Will mark ready for review once KVM releases new version, which should include newly added KVM_X86_SET_MSR_FILTER vm ioctl that this PR depends on, see rust-vmm/kvm#359

@ludfjig
ludfjig force-pushed the reset2 branch 7 times, most recently from aeca04d to 968f2f4 Compare October 31, 2025 00:29
@ludfjig ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Dec 2, 2025
@ludfjig
ludfjig force-pushed the reset2 branch 3 times, most recently from e859ff9 to e793129 Compare December 8, 2025 18:42
@ludfjig ludfjig changed the title Reset more state when restoring snapshot Reset MSRs when restoring snapshot Dec 18, 2025
@ludfjig ludfjig changed the title Reset MSRs when restoring snapshot Disallow MSR read/writes Feb 10, 2026
@ludfjig
ludfjig force-pushed the reset2 branch 5 times, most recently from 8f55249 to 3bb2294 Compare February 13, 2026 23:16
@ludfjig
ludfjig force-pushed the reset2 branch 2 times, most recently from 032168d to c0383dd Compare February 18, 2026 19:27
@ludfjig
ludfjig force-pushed the reset2 branch 6 times, most recently from 2e98073 to 089227c Compare July 11, 2026 07:10
@ludfjig
ludfjig force-pushed the reset2 branch 11 times, most recently from 99c2a45 to a4cf1de Compare July 17, 2026 04:06
@ludfjig ludfjig changed the title Disallow MSR read/writes Prevent MSRs from leaking across calls to MultiuseSandbox::restore Jul 17, 2026
@ludfjig
ludfjig marked this pull request as ready for review July 17, 2026 04:12
Copilot AI review requested due to automatic review settings July 17, 2026 04:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR prevents x86_64 model-specific register (MSR) state from persisting across snapshot restores by adding explicit MSR capture/reset semantics to snapshots, enforcing an MSR allow-list contract, and (on KVM) denying guest MSR access by default via an MSR filter.

Changes:

  • Capture MSR reset state into Snapshot (and snapshot-on-disk format), and restore it during MultiUseSandbox::from_snapshot / MultiUseSandbox::restore.
  • Add SandboxConfiguration::allow_msrs (bounded allow list) and enforce allow-list compatibility on restore (destination must be a superset).
  • Implement backend-specific MSR handling: KVM deny-by-default filtering + violation reporting; MSHV/WHP MSR read/write support for reset.

Reviewed changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/tests/rust_guests/simpleguest/src/main.rs Adds guest-side RDMSR/WRMSR and SWAPGS helpers for host MSR tests.
src/hyperlight_host/src/sandbox/snapshot/mod.rs Extends Snapshot with optional captured MSRs and snapshot allow list.
src/hyperlight_host/src/sandbox/snapshot/file/mod.rs Persists/loads MSR fields and validates msrs/allowed_msrs consistency.
src/hyperlight_host/src/sandbox/snapshot/file/config.rs Adds MSR fields to OCI snapshot config and serde tests/schema pin updates.
src/hyperlight_host/src/sandbox/snapshot/file_tests.rs Adds end-to-end disk snapshot tests for MSR capture/compat/allow-list rules.
src/hyperlight_host/src/sandbox/initialized_multi_use.rs Captures MSRs on snapshot and restores them on from_snapshot/restore + extensive MSR tests.
src/hyperlight_host/src/sandbox/config.rs Introduces MSR allow-list storage and capacity-checked allow_msrs.
src/hyperlight_host/src/mem/mgr.rs Threads MSR state through snapshot construction.
src/hyperlight_host/src/hypervisor/virtual_machine/whp.rs Adds WHP MSR register mapping + get/set MSRs for reset.
src/hyperlight_host/src/hypervisor/virtual_machine/mshv/x86_64.rs Adds MSHV MSR mapping + get/set MSRs for reset.
src/hyperlight_host/src/hypervisor/virtual_machine/mod.rs Extends VirtualMachine with MSR read/write APIs + validation helpers/errors.
src/hyperlight_host/src/hypervisor/virtual_machine/kvm/x86_64.rs Implements KVM MSR filter configuration, filtered-exit handling, and KVM MSR get/set.
src/hyperlight_host/src/hypervisor/regs/x86_64/special_regs.rs Adds test asserting x2APIC stays disabled by default.
src/hyperlight_host/src/hypervisor/regs/x86_64/msrs.rs New MSR reset table + snapshot validation logic and related constants/tests.
src/hyperlight_host/src/hypervisor/regs/x86_64/mod.rs Exposes the new MSR module via pub(crate) use.
src/hyperlight_host/src/hypervisor/hyperlight_vm/x86_64.rs Captures baseline MSR reset state at VM creation and implements restore logic.
src/hyperlight_host/src/hypervisor/hyperlight_vm/mod.rs Wires KVM MSR violations into RunVmError and stores msr_reset in HyperlightVm.
src/hyperlight_host/src/error.rs Adds HyperlightError::{MsrReadViolation, MsrWriteViolation} and marks them poisoning on KVM.
Justfile Adds ignored host-dependent MSR audit test recipes to test-isolated.
docs/msr.md Documents MSR reset set, backend differences, validation rules, and limitations.
CHANGELOG.md Adds breaking-change entry for MSR allow-list + restore behavior.
.github/workflows/ValidatePullRequest.yml Disables matrix fail-fast for PR validation workflow.

Comment thread src/hyperlight_host/src/sandbox/snapshot/mod.rs Outdated
Comment thread docs/msr.md Outdated
Comment thread CHANGELOG.md Outdated
Comment thread src/hyperlight_host/src/sandbox/snapshot/file/config.rs
Comment thread src/hyperlight_host/src/sandbox/snapshot/file/config.rs
KVM denies guest MSR access by default. SandboxConfiguration::allow_msrs permits selected MSRs.

MSHV and WHP have no per-MSR filter. Hyperlight captures exposed retained MSR state at VM creation and resets it on restore.

Captured MSR state persists in OCI snapshots. KVM denials report the MSR index. Unsupported accesses on MSHV and WHP raise a guest general protection fault. Both failures poison the sandbox.

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@ludfjig ludfjig added the ready-for-review PR is ready for (re-)review label Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. ready-for-review PR is ready for (re-)review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants