Skip to content

feat: subprocess support, networking idle-poll fix, and CI improvements#108

Merged
danbugs merged 18 commits into
mainfrom
feat/subprocess-support
Jul 3, 2026
Merged

feat: subprocess support, networking idle-poll fix, and CI improvements#108
danbugs merged 18 commits into
mainfrom
feat/subprocess-support

Conversation

@danbugs

@danbugs danbugs commented Jul 3, 2026

Copy link
Copy Markdown
Contributor

Add subprocess support for the Hyperlight platform via vfork/execve, enabling
Python's subprocess.run() to execute busybox commands and pip installs inside
the guest. Since Hyperlight has no MMU, fork() is promoted to vfork
(CLONE_VM | CLONE_VFORK) — the child shares the parent's address space.

  • Increase heap to 2.5 GiB for subprocess memory requirements
  • Point python-agent-driver kraft.yaml at upstream unikraft/app-elfloader plat-hyperlight
  • Add busybox and pip install demo scripts with CI regression tests
  • Build base Docker images automatically in just rootfs
  • Replace strtoul with inline hex parser (no libc dependency in hl_pydriver)
  • Add urllib GET without timeout test for idle-poll networking path
  • Add per-test timeout wrapper to Windows CI runtime tests

Copilot AI review requested due to automatic review settings July 3, 2026 01:38

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Python agent driver example and host tooling to better support subprocess-style workflows (pip installs and running basic shell utilities inside the guest), along with a small adjustment to socket polling behavior during guest sleep.

Changes:

  • Narrow poll/WSAPoll event masks used by __hl_sleep socket polling while sleeping.
  • Increase the warm-up sandbox heap size used during pyhl setup.
  • Update the python-agent-driver example rootfs and build inputs to support pip installs (offline wheels) and BusyBox-based subprocess demos.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
host/src/lib.rs Adjusts socket poll event masks used during sleep to avoid polling for writability.
host/src/bin/pyhl.rs Increases the fixed heap size used for the warm-up/snapshot sandbox during pyhl setup.
examples/python-agent-driver/kraft.yaml Switches Unikraft/app-elfloader sources to a fork/feature branch and enables additional process/FD config.
examples/python-agent-driver/hl_pydriver.c Adds hex parsing helper and changes how kernel-provided pointer env vars are parsed.
examples/python-agent-driver/Dockerfile Adds offline wheel download, restores pip + needed stdlib pieces, and adds BusyBox + supporting rootfs setup.
examples/python-agent-driver/demo_pip_install.py Adds a demo that installs/imports six via pip at runtime in the guest.
examples/python-agent-driver/demo_busybox.py Adds a demo that runs basic BusyBox commands via subprocess in the guest.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread host/src/bin/pyhl.rs
Comment thread examples/python-agent-driver/kraft.yaml Outdated
Comment thread examples/python-agent-driver/kraft.yaml Outdated
Comment thread examples/python-agent-driver/hl_pydriver.c
Comment thread examples/python-agent-driver/Dockerfile Outdated
Comment thread examples/python-agent-driver/Dockerfile
Comment thread examples/python-agent-driver/demo_pip_install.py

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Linux Benchmarks

Details
Benchmark suite Current: 8084df2 Previous: 682ad20 Ratio
hello_world (median) 20 ms 20 ms 1
pandas (median) 110 ms 120 ms 0.92
density (per VM) 11 MB 8 MB 1.38
snapshot (disk) 702 MiB 653 MiB 1.08

This comment was automatically generated by workflow using github-action-benchmark.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Windows Benchmarks

Details
Benchmark suite Current: 8084df2 Previous: 682ad20 Ratio
hello_world (median) 354 ms 291 ms 1.22
pandas (median) 1095 ms 1024 ms 1.07
density (per VM) 11 MB 7 MB 1.57
snapshot (disk) 728 MiB 661 MiB 1.10

This comment was automatically generated by workflow using github-action-benchmark.

danbugs added 6 commits July 3, 2026 15:38
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
@danbugs danbugs force-pushed the feat/subprocess-support branch from e84b5e6 to 89c6afb Compare July 3, 2026 16:19
danbugs added 3 commits July 3, 2026 16:47
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
@danbugs danbugs changed the title bump and test feat: subprocess support, networking idle-poll fix, and CI improvements Jul 3, 2026
danbugs added 9 commits July 3, 2026 16:55
…demo

Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
Signed-off-by: danbugs <danilochiarlone@gmail.com>
hyperlight-unikraft writes guest output to stderr with -q flag.
The Linux test merges stderr via 2>&1, but the Windows helper only
read stdout. Merge both streams so output matching works on Windows.

Signed-off-by: danbugs <danilochiarlone@gmail.com>
PR unikraft/unikraft#1860 (epoll bypass guard) is now merged.

Signed-off-by: danbugs <danilochiarlone@gmail.com>
@danbugs danbugs merged commit 6534cf1 into main Jul 3, 2026
89 checks passed
@danbugs danbugs deleted the feat/subprocess-support branch July 3, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants