Skip to content

fix(appimage): unblock release smoke by relaxing the userns sysctl - #5251

Merged
M3gA-Mind merged 1 commit into
tinyhumansai:releasefrom
M3gA-Mind:fix/appimage-smoke-userns-DAPPIMG-39513
Jul 28, 2026
Merged

fix(appimage): unblock release smoke by relaxing the userns sysctl#5251
M3gA-Mind merged 1 commit into
tinyhumansai:releasefrom
M3gA-Mind:fix/appimage-smoke-userns-DAPPIMG-39513

Conversation

@M3gA-Mind

Copy link
Copy Markdown
Contributor

Release-blocking. Base is release, not main. Every Release Production run has failed on the Linux desktop job since #5189 reached release.
This must also be cherry-picked to main so main does not regress the moment it is promoted again.
Failed run: https://github.com/tinyhumansai/openhuman/actions/runs/30361949794 (job Build desktop matrix / Desktop: ubuntu, release @ 9c1e2e65)

Summary

  • Unblocks Release Production: the Linux x86_64 AppImage startup smoke added by fix(appimage): anchor sharun library paths to AppDir #5189 has never passed and fails deterministically with exit 133 instead of the expected 124.
  • Root cause is Ubuntu 23.10+'s kernel.apparmor_restrict_unprivileged_userns=1, which denies unprivileged user namespaces to unconfined processes, so CEF's zygote aborts with No usable sandbox!.
  • fix(appimage): anchor sharun library paths to AppDir #5189's mitigation — a per-executable AppArmor profile — loads successfully but never takes effect, because AppArmor attaches profiles by execve path and the sharun launcher does not exec shared/bin/OpenHuman directly.
  • Applies Chromium's other documented remedy instead: relax the sysctl for the smoke window and restore it afterwards. Path-independent, so it cannot miss the way profile attachment does.
  • No workflow change and no change to the validator's public contract; the fix is entirely inside validate-appimage-runtime.sh and its fixture tests.

Problem

The Strip host graphics libs and validate final AppImage step fails on ubuntu (x86_64). The AppImage itself is fine — it builds, signs, bundles libcef.so, and boots far enough to log [cef-startup]. What fails is the new bounded startup smoke:

[appimage-runtime] AppImage startup smoke failed (status 133):
[appimage-runtime]   unshare: write failed /proc/self/uid_map: Operation not permitted
[appimage-runtime]   ...
[appimage-runtime]   [59873:59873:...:FATAL:content/browser/zygote_host/zygote_host_impl_linux.cc:128]
                     No usable sandbox! If you are running on Ubuntu 23.10+ or another Linux distro
                     that has disabled unprivileged user namespaces with AppArmor ...
[appimage-runtime]   Trace/breakpoint trap (core dumped)

smoke_extracted_apprun treats timeout's status 124 as success. CEF dies on SIGTRAP about one second in, so the smoke sees 133 and fails the release.

This is a regression from #5189, and the smoke has never passed

Run Head SHA Result
30240103654 e3a7f5217 ✅ last green — pre-#5189 script
30242402885 2e5b5e7b2Promote main → release (#5203), which carried #5189
30351759443 a40fbb79d
30361949794 9c1e2e657

The failure is byte-identical across all three. #5189 merged into main, where the production AppImage build does not run, so this was never exercised before promotion. ubuntu-arm64 is unaffected only because build-desktop.yml enables the smoke for x86_64 alone.

Why #5189's AppArmor profile does not work

install_smoke_userns_profile implements Chromium's per-executable-profile remedy against $appdir/shared/bin/OpenHuman. The profile loads — the log confirms Loaded temporary AppArmor userns profile for: .../shared/bin/OpenHuman — yet unshare is still denied. AppArmor attaches profiles by the execve'd path, and the sharun launcher runs the app through the AppDir's bundled dynamic loader rather than exec'ing shared/bin/OpenHuman; Chromium then re-execs /proc/self/exe for the zygote. The confinement target never matches, so the userns, grant is never in force.

Solution

In scripts/release/validate-appimage-runtime.sh:

  • Add relax_smoke_userns_restriction / restore_smoke_userns_restriction, which set kernel.apparmor_restrict_unprivileged_userns=0 for the smoke window and restore the host's original value afterwards — including on the existing HUP/INT/TERM interrupt path and when AppArmor setup itself fails. The runner is left exactly as it was found.
  • The toggle is a no-op when the key is already 0 or absent on the host, so it never reaches for sudo outside the restricted-Ubuntu case it exists for.
  • The AppArmor profile is retained alongside the sysctl: it is harmless, and it keeps the narrower per-executable grant in place for hosts where the sysctl is unavailable.
  • The four-argument contract of smoke_extracted_apprun_with_userns is unchanged; the saved-state file is derived as $profile_file.sysctl. No change to build-desktop.yml is required.

Resulting privileged-command sequence, pinned by fixtures:

sudo sysctl -q -w …userns=0 → apparmor_parser --replace → smoke → apparmor_parser --remove → sudo sysctl -q -w …userns=<original>

Rejected alternative: OPENHUMAN_CEF_NO_SANDBOX / --no-sandbox

The smoke currently unsets OPENHUMAN_CEF_NO_SANDBOX, so setting it looks like the obvious fix. It is not:

Keeping the sandbox enabled is also the better outcome — the smoke goes on exercising the real production configuration rather than a weakened one.

Impact

  • CI/release only. No shipped code changes: no Rust, no frontend, no bundling behaviour, no change to the AppImage bytes.
  • Security posture is unchanged in the product. The sysctl is relaxed only on the ephemeral hosted runner, only for the ~15-second smoke window, and is restored afterwards; Chromium's sandbox stays enabled in the smoked app.
  • If a maintainer needs a second escape hatch, APPIMAGE_RUNTIME_SMOKE: '0' for x86_64 in .github/workflows/build-desktop.yml disables the gate in one line.

Verification

scripts/release/test-strip-appimage-rpaths.sh is Linux-only and skips on macOS (SKIP: patchelf not installed), so the new logic was additionally driven directly against the same command fakes:

  • restricted host → relax, save 1, restore 1, state file removed;
  • permissive host (0) → no sudo, no state file;
  • absent sysctl key → no sudo, no state file;
  • full wrapper ordering, preservation of a failing smoke's status 37, and no leaked state.

The observed sequence matches the fixture expectations exactly. This cannot be validated end-to-end by normal PR CI — see the note below.

Related

  • Regressed by: fix(appimage): anchor sharun library paths to AppDir #5189 (reached release via promote Promote main → release #5203)
  • Follow-up PR(s)/TODOs:
    • Cherry-pick this commit to main so the next promotion does not reintroduce the failure.
    • Consider whether install_smoke_userns_profile should be dropped entirely once a green run confirms the sysctl is the effective remedy; if kept, retarget it at the sharun execve path so the grant actually attaches.
    • Consider running the production AppImage build on PRs that touch scripts/release/**, so a smoke gate cannot merge unexercised again.
  • Closes:

Verification by a maintainer is required

Regular PR CI does not run the production Linux AppImage build, so this PR's checks cannot prove the fix. A maintainer must re-dispatch the Release Production workflow (against release) to confirm the Desktop: ubuntu job now reaches status 124 and passes.

Submission Checklist

  • Tests added or updated (happy path + at least one failure / edge case) per Testing Strategy — test-strip-appimage-rpaths.sh: updated both expected privileged-command-order fixtures (failing-smoke and TERM-interrupt paths), added no-op coverage for permissive-host and absent-key, and assertions that the saved-state file never leaks.
  • Diff coverage ≥ 80% — changed lines are Bash in scripts/release/**, covered by the test-strip-appimage-rpaths.sh fixture suite. diff-cover over Vitest + cargo-llvm-cov does not instrument shell; no TS/Rust lines changed.
  • Coverage matrix updated — N/A: CI/release tooling change, no product feature row affected.
  • All affected feature IDs from the matrix are listed under ## RelatedN/A: no feature IDs affected.
  • No new external network dependencies introduced — no new downloads or endpoints; sysctl/sudo are already required by this script.
  • Manual smoke checklist updated if this touches release-cut surfaces — N/A: restores the intended behaviour of an existing automated release gate; the manual checklist is unchanged.
  • Linked issue closed via Closes #NNNN/A: no tracking issue; this PR is driven by the failed release run linked above.

AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: fix/appimage-smoke-userns-DAPPIMG-39513
  • Commit SHA: 0be92171b

Validation Run

  • pnpm --filter openhuman-app format:checkN/A: no files under app/ changed.
  • pnpm typecheckN/A: no TypeScript changed.
  • Focused tests: bash -n on both scripts; new relax/restore functions and smoke_extracted_apprun_with_userns driven against the fixture command fakes (restricted / permissive / absent-key / wrapper ordering / failure-status preservation). scripts/release/test-strip-appimage-rpaths.sh skips on macOS (patchelf is Linux-only) and runs in CI.
  • Rust fmt/check (if changed): N/A: no Rust changed.
  • Tauri fmt/check (if changed): N/A: no Tauri code changed.

Validation Blocked

  • command: Release ProductionBuild desktop matrix / Desktop: ubuntu
  • error: not runnable locally — Linux-only AppImage bundling plus an Xvfb/AppArmor runtime smoke; this work was done on macOS, and the job is not triggered by regular PR CI.
  • impact: the end-to-end fix can only be confirmed by a maintainer re-dispatching Release Production. Diagnosis was derived from the full failing log and the scripts; the new shell logic is verified locally against command fakes.

Behavior Changes

  • Intended behavior change: during the release AppImage smoke only, unprivileged user namespaces are permitted on the runner so CEF's zygote sandbox can initialise; the host value is restored immediately afterwards.
  • User-visible effect: none. No shipped artifact changes.

Parity Contract

  • Legacy behavior preserved: the AppArmor profile install/remove path, the four-argument signature of smoke_extracted_apprun_with_userns, the 124-means-alive success rule, the forbidden-loader-diagnostic greps, secret scrubbing, and signal-trap restore are all unchanged.
  • Guard/fallback/dispatch parity checks: the sysctl toggle is skipped entirely when the key reads 0 or is absent, so non-Ubuntu and already-permissive hosts follow exactly the previous code path; restore is idempotent and runs on the success, failure, AppArmor-setup-failure, and interrupt paths.

Duplicate / Superseded PR Handling

  • Duplicate PR(s): none
  • Canonical PR: this one
  • Resolution: N/A

The Linux desktop job has failed every Release Production run since tinyhumansai#5189
reached release (promote tinyhumansai#5203). The AppImage itself builds, signs and boots
fine; the new bounded startup smoke is what fails, deterministically, with
exit 133 instead of the expected 124:

    FATAL:content/browser/zygote_host/zygote_host_impl_linux.cc:128]
    No usable sandbox! If you are running on Ubuntu 23.10+ ...

Ubuntu 23.10+ sets kernel.apparmor_restrict_unprivileged_userns=1, which
denies unprivileged user-namespace creation to every unconfined process on
the runner, so CEF's zygote cannot build its sandbox and aborts on SIGTRAP.

tinyhumansai#5189 reached for Chromium's second documented remedy, a per-executable
AppArmor profile carrying `userns,`. The profile loads successfully but never
takes effect, because AppArmor attaches profiles by execve path while the
sharun launcher runs the app through the AppDir's bundled dynamic loader
rather than exec'ing shared/bin/OpenHuman directly (Chromium then re-execs
/proc/self/exe for the zygote).

Apply Chromium's first documented remedy instead: relax the sysctl for the
smoke window and restore its original value afterwards, including on
interrupt. It is path-independent, so it cannot miss the way profile
attachment does. The AppArmor profile is retained alongside it -- it is
harmless and keeps the narrower grant for hosts without the sysctl.

Setting OPENHUMAN_CEF_NO_SANDBOX is deliberately not the fix: it is
#[cfg(debug_assertions)]-gated in app/src-tauri/src/lib.rs and therefore
inert in the release AppImage, and CEF's command line is built explicitly
via command_line_args() so a --no-sandbox argv flag would not reach it
either. Keeping the sandbox enabled also means the smoke keeps exercising
the real production configuration.

The toggle is a no-op on hosts that are already permissive or that have no
such sysctl key, so it never reaches for sudo outside the restricted-Ubuntu
case it exists for.

Refs: https://github.com/tinyhumansai/openhuman/actions/runs/30361949794
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9c35abf4-6022-4b1b-ba12-a04ae7ea46cd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

@M3gA-Mind
M3gA-Mind merged commit 42cdf77 into tinyhumansai:release Jul 28, 2026
33 of 36 checks passed
M3gA-Mind added a commit that referenced this pull request Jul 28, 2026
…ain copy of #5251) (#5252)

Co-authored-by: M3gA-Mind <elvin@mahadao.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0be92171b9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +256 to +259
sudo --non-interactive sysctl -q -w "$SMOKE_USERNS_SYSCTL=0" \
|| { runtime_validation_error "could not relax $SMOKE_USERNS_SYSCTL for the AppImage smoke"; return 1; }

printf '%s\n' "$current" >"$previous_file"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Record sysctl state before relaxing userns

When the AppImage smoke runs on a restricted Ubuntu host and the job is cancelled or TERM'd just after sudo sysctl ...=0 takes effect but before $previous_file is written, the installed signal trap calls restore_smoke_userns_restriction with no state file and therefore skips the restore. That leaves unprivileged user namespaces enabled for the rest of the runner/session; write the previous value before flipping the sysctl (or otherwise mark that a restore is required) so the interrupt path can always put the host back.

Useful? React with 👍 / 👎.

@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown

Greptile Summary

This PR fixes a deterministic release-blocking failure on the Ubuntu AppImage smoke introduced by #5189. The root cause is Ubuntu 23.10+'s kernel.apparmor_restrict_unprivileged_userns=1 sysctl, which denies unprivileged user-namespaces to unconfined processes; CEF's zygote therefore aborts with No usable sandbox!, producing exit 133 instead of the expected 124.

  • validate-appimage-runtime.sh: Adds relax_smoke_userns_restriction / restore_smoke_userns_restriction that toggle the sysctl to 0 for the smoke window and restore the original value afterwards. Both the normal completion path and the HUP/INT/TERM interrupt path call restore; early-return guards make the toggle a no-op when the key is absent or already 0. The existing AppArmor per-executable profile is retained alongside the sysctl fix.
  • test-strip-appimage-rpaths.sh: Adds a fake sysctl binary to the existing command-fake harness, updates the two command-order fixture assertions to include the sysctl relax/restore calls, and introduces new no-op coverage for permissive-host (value 0) and absent-key cases.

Confidence Score: 5/5

Safe to merge — changes are confined to CI release tooling with no effect on shipped artifact bytes, and all new code paths are covered by the fixture test suite.

The two new helpers correctly handle the absent-key, already-permissive, relax-failure, and restore-failure cases. Cleanup is wired into all three exit paths (success, AppArmor-setup failure, and the HUP/INT/TERM interrupt). The state-file-as-mutex pattern prevents double-restore races, the toggle is fully exercised by updated command-order fixtures, and no-op coverage for permissive/absent-key hosts is new. No shipped code changes.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
scripts/release/validate-appimage-runtime.sh Adds sysctl relax/restore helpers and integrates them into smoke_extracted_apprun_with_userns with correct cleanup on success, AppArmor-setup failure, and signal-interrupt paths; state-file-as-mutex prevents double-restore races.
scripts/release/test-strip-appimage-rpaths.sh Adds fake sysctl binary to command-fake harness, updates two command-order fixture assertions, and adds no-op coverage for permissive (value=0) and absent-key cases; state-file non-leakage asserted on both failure and TERM paths.

Reviews (1): Last reviewed commit: "fix(appimage): unblock release smoke by ..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants