Guest Python (penguest) + first-class uncompiled init drop-ins

[lacraig2]

Summary

Implements two coupled planning drafts:

• Draft 24 — first-class uncompiled init drop-ins. .sh/.py files (and extension-less scripts with a shell/python shebang) dropped in init.d/ are now treated like the existing .c compileables: their shebang is normalized to a guest interpreter (/igloo/utils/sh, /igloo/utils/python3), a foreign/missing one is rewritten with a warning, and the body is installed into /igloo/init.d/. Non-scripts (prebuilt binaries, .conf/.txt, deliberate foreign shebangs, busybox multi-call shebangs) stay verbatim.

• Draft 16 — guest Python + host comms. A guest-side penguest Python module (staged into /igloo/pylib/penguest/, on PYTHONPATH via the python3 wrapper) that wraps:

  • portal_call(magic, *args) — the existing portalcall sendto ABI (per-arch syscall table), plus typed log() / report() helpers.
  • penguest.vsock — a framed length-prefixed-JSON AF_VSOCK client to the host.

  Host side (both in default_plugins): apis/penguest.py (log bridge → run log + persisted penguest_guest.log) and apis/penguest_vsock.py (vsock endpoint, ping/echo built in, @endpoint registration).

The two are coupled: .py drop-ins only make sense because in-guest CPython is staged, and import penguest works with no extra setup.

Security

The guest is untrusted firmware, so these are host-facing channels. portal_call adds no new transport (the raw sendto was always available). The two new default handlers take untrusted guest input and are hardened accordingly:

• log bridge: 64 KiB read cap, control-char sanitization (no log-line forgery / terminal-escape injection), bounded log file.
• vsock endpoint: capped worker pool (vsock_max_conns) with a per-connection recv timeout (vsock_conn_timeout) so a stalled/flooding guest can't wedge it, 4 MiB frame cap, deterministic teardown.

Disable per run for adversarial-firmware analysis:

plugins:
  penguest: { enabled: false }
  penguest_vsock: { enabled: false }

penguest_vsock is inherently a no-op when vsock/vpn is off.

Testing

• Unit (tests/unit/, per the test-harness: host-side pyplugin testing (pyproject + CI fast lane + in-place harness) #881 host-testing contract): test_init_dropins.py (shebang resolution + end-to-end config load) and test_penguest.py (portal_call packing vs portal_call.h, a round-trip through the real portalcall.py, vsock JSON framing, log/report packing, the host bridge reading guest memory + persisting/bounding the log, and the vsock endpoint incl. a live listener, connection cap, and stalled-connection drop). Full suite: 636 passed, flake8 clean, verified in a fresh venv.
• Integration (tests/integration/test_target/patches/tests/penguest.yaml, default arch matrix): a .py driver run via its #!/igloo/utils/python3 shebang that import penguest, makes a portal_call, calls log/report, and does a vsock.connect() echo round-trip through vhost-device-vsock — the one place the guest→host forwarding is exercised end-to-end on real guest CPython.

Note: the 32-bit guest-python3 hang/ENOSYS (penguin#876) is fixed in the pinned penguin-tools via penguin-tools#20's glibc vDSO runtime-gate (mipsel/mipseb/armel); 64-bit arches were never affected.

Docs

docs/init_dropins.md, docs/penguest.md (incl. Security section), regenerated docs/schema_doc.md.