Feature Request
Automatically capture a guest vmcore on kernel panic via a host-side pvpanic device, without reserving any guest memory.
Today the only supported way to obtain a full memory image of a panicked guest kernel is in-guest kdump/kexec, which requires reserving memory via the crashkernel= kernel command line parameter — in practice a fixed ~100–200 MB per VM. For microVMs, which are often provisioned with only a few hundred MB of RAM, that reservation consumes a large fraction of guest memory.
But the VMM already has the entire guest physical memory mapped. If the guest signals the host on panic, the host can produce a complete crash dump itself, with zero guest-side memory reservation.
Describe the desired solution
Add an opt-in, per-VM paravirtualized pvpanic device. The guest-side driver is already in mainline Linux (CONFIG_PVPANIC), so the guest needs no changes, it simply signals the device on panic.
When Firecracker receives that event, it automatically writes a vmcore to a configured host path that is directly analyzable with crash/gdb, with no in-guest crashkernel= reservation.
Describe possible alternatives
- In-guest kdump via
crashkernel=. Works, but requires a fixed ~100–200 MB reservation per guest.
- Serial-console panic logs only. No memory reservation, but only yields the backtrace/log text.
Additional context
I have a working x86_64 PoC (the pvpanic device plus the host-side ELF vmcore dumper) that produces crash/gdb-analyzable dumps, with negligible impact on Firecracker's process RSS.
However, I'm not sure whether a full guest vmcore dump really fits Firecracker's scope, given the minimalist core tenet, or whether a lighter mechanism would be preferred. I also noticed that pvpanic config isn't enabled in the CI kernels.
If there's a better way to dump a panicked kernel without adding a new device, it would also be much appreciated. And if the community is interested in this feature, I'm very happy to open a PR for this.
Checks
Feature Request
Automatically capture a guest
vmcoreon kernel panic via a host-sidepvpanicdevice, without reserving any guest memory.Today the only supported way to obtain a full memory image of a panicked guest kernel is in-guest kdump/kexec, which requires reserving memory via the
crashkernel=kernel command line parameter — in practice a fixed ~100–200 MB per VM. For microVMs, which are often provisioned with only a few hundred MB of RAM, that reservation consumes a large fraction of guest memory.But the VMM already has the entire guest physical memory mapped. If the guest signals the host on panic, the host can produce a complete crash dump itself, with zero guest-side memory reservation.
Describe the desired solution
Add an opt-in, per-VM paravirtualized
pvpanicdevice. The guest-side driver is already in mainline Linux (CONFIG_PVPANIC), so the guest needs no changes, it simply signals the device on panic.When Firecracker receives that event, it automatically writes a
vmcoreto a configured host path that is directly analyzable withcrash/gdb, with no in-guestcrashkernel=reservation.Describe possible alternatives
crashkernel=. Works, but requires a fixed ~100–200 MB reservation per guest.Additional context
I have a working x86_64 PoC (the
pvpanicdevice plus the host-side ELFvmcoredumper) that producescrash/gdb-analyzable dumps, with negligible impact on Firecracker's process RSS.However, I'm not sure whether a full guest vmcore dump really fits Firecracker's scope, given the minimalist core tenet, or whether a lighter mechanism would be preferred. I also noticed that
pvpanicconfig isn't enabled in the CI kernels.If there's a better way to dump a panicked kernel without adding a new device, it would also be much appreciated. And if the community is interested in this feature, I'm very happy to open a PR for this.
Checks