Skip to content

fix(studio): serve container-baked assets, and stop shipping a partial Studio bundle [ASTD-354] - #1080

Merged
marcusds merged 4 commits into
mainfrom
astd-354-studio-assets-container/mschwab
Aug 4, 2026
Merged

fix(studio): serve container-baked assets, and stop shipping a partial Studio bundle [ASTD-354]#1080
marcusds merged 4 commits into
mainfrom
astd-354-studio-assets-container/mschwab

Conversation

@marcusds

@marcusds marcusds commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Problem

Running nvcr.io/nvidia/nemo-platform/nmp-api directly — docker run, compose, quickstart, anything that is not the Helm chart — lands on the error page at /studio:

NeMo Studio assets are not built
Expected assets at: /app/.venv/lib/python3.13/site-packages/nmp/studio/static

Helm deployments are unaffected because k8s/helm/values.yaml sets studio.static_files_path: "/static/studio" explicitly, masking the gap.

Investigating this turned up a second, independent problem: the arm64 image at that tag ships a Studio bundle with no index.html at all. Three bugs in total — the assets were unreachable (1), a broken bundle could ship green (2), and the build chain that produced the broken bundle should never have been running (3).

Bug 1 — the container bundle was never consulted

docker/Dockerfile.nmp-api copies the built bundle to /static/studio, but sets no config pointing at it. StudioConfig.static_files_path defaults to None, so _get_static_files_path falls back to the packaged static/ dir beside the nmp.studio package — which is only populated by the wheel build (packages/nemo_platform/pyproject.toml force_include). The container builder stage runs uv sync without web/dist present, so it stays empty. Nothing ever looked at /static/studio.

Fix: add /static/studio to the _get_static_files_path fallback chain — configured → packaged → container → source checkout.

Why not bake NMP_STUDIO_STATIC_FILES_PATH into the Dockerfile

That was the obvious one-line fix, and it is wrong here. ServiceConfig derives from EnvironmentFirstSettings (packages/nemo_platform_plugin/src/nemo_platform_plugin/config.py:143-160), whose settings_customise_sources returns (env_settings, dotenv_settings, init_settings, file_secret_settings)env ahead of init kwargs, the reverse of the pydantic-settings default. So the env var would silently override an operator's studio.static_files_path. Confirmed:

os.environ["NMP_STUDIO_STATIC_FILES_PATH"] = "/static/studio"
Configuration.global_settings_to_service_config(
    {"studio": {"static_files_path": "/operator/custom"}}, StudioConfig
).static_files_path
# -> /static/studio

The fallback chain fixes this image, covers future images that lay the bundle down without setting a var, and cannot shadow config. docker/Dockerfile.nmp-api is untouched. A regression test pins the env-over-YAML ordering so nobody re-adds the env var later.

Bug 2 — the arm64 image ships a public-only dist/

/static/studio in the arm64 image at that tag contains exactly favicon.svg, sample-agents/, sample-datasets/ — byte-for-byte the contents of web/packages/studio/public/. No index.html, no assets/. 877 kB.

It is arch-specific. Comparing the two manifests layer-by-layer, every layer matches within a few percent except one:

layer 14 (compressed) amd64 arm64
2,583,129 B 293,216 B

Why public-only: Vite 8 copies publicDiroutDir from the vite:prepare-out-dir renderStart hook — before chunks are rendered and written. Kill the build during chunk rendering and dist/ holds public files and nothing else. The Dockerfile's own comment names that failure mode: "QEMU arm64 builds occasionally hang in Vite chunk rendering."

Why it did not fail the builddocker/base/Dockerfile.nmp-studio-ui:

if ... timeout --kill-after=30s 15m pnpm ... build:fastapi; then
  exit 0;
fi;
status="$?";                                    # <-- always 0
if [ "${status}" != "124" ] && [ "${status}" != "137" ]; then
  exit "${status}";                             # <-- exit 0
fi;

$? after fi is the status of the if compound command, not the failed build — a failing condition with no else leaves the if itself at 0. So a SIGKILL at the 15m timeout produced 124, was captured as 0, fell through 0 != 124 && 0 != 137, and ran exit 0. The RUN succeeded, the three-attempt retry never once fired, and the partial dist/ shipped.

Fix (backstop): capture the status with || status="$?" outside the if, and assert dist/index.html exists before reporting success. That converts a silently broken image into a failed build — necessary, but it only makes the failure loud.

Bug 3 — the emulated build chain should not exist

Making the build fail loudly would have turned arm64 release builds red rather than green-and-broken. The actual fix is to remove the emulation.

nmp-studio-ui declares platforms = [linux/amd64, linux/arm64], so buildkit runs the entire node + pnpm + vite chain twice and emulates whichever half does not match the builder. But dist/ is architecture-independent JS/CSS/HTML — the second chain produces the same bytes and contributes nothing except the failure mode.

Fix: pin the node stages to the builder's native arch.

FROM --platform=$BUILDPLATFORM ${DOCKERHUB_MIRROR}/node:${NODE_VERSION}-bookworm AS base

$BUILDPLATFORM is whatever the builder host is, so this is symmetric — an amd64 runner builds amd64 natively, an arm64 runner builds arm64 natively, and neither emulates. The FROM scratch AS artifacts stage is still stamped per target platform, so nmp-api consumes it unchanged. Also collapses the expensive pnpm install + vite build from twice to once.

nmp-studio-ui has no cache-to/cache-from (unlike nmp-api-docker) and its only consumer is the nmp-api-docker named context, so there are no cache implications.

Also in scope

The error page's recovery block printed nvm / make bootstrap-studio / nemo services restart unconditionally — meaningless inside a container, and it sent the reporting user down the wrong path. It now branches on whether a source checkout is detected.

Source checkouts keep the build tips. Packaged installs get no remediation steps at all, because a packaged install ships with the bundle and there is nothing for the operator to build — a missing bundle there is a packaging defect, not a setup mistake. They get a plain error plus a link to the docs site, the only outward-facing help pointer the repo has (README badge, web/packages/studio/src/constants/links.ts):

<p>Expected assets at: <code>/static/studio</code></p>
<p>This install ships with the Studio bundle, so this is unexpected.</p>
<p>See the <a href="https://docs.nvidia.com/nemo-platform">NeMo Platform documentation</a> for help.</p>

Heading changed from "assets are not built" to "assets were not found".

Verification

Bug 1, against a real nmp-api image with the patched nmp/studio package bind-mounted:

  • before: falls through, no SPA
  • after, no config file: /studio/200 text/html, SPA index served
  • config file setting studio.static_files_path → resolves to the configured path; container fallback loses
  • error page rendered in-container shows the packaged-install notice and docs link, not build tips

Bug 2, real builds of the nmp-studio-ui target:

  • happy path — ✓ built in 3.49s, 12.28 MB exported, /artifacts has index.html + assets/ (331 files). Exit 0.
  • guard path — same build with && rm -f dist/index.html spliced in:
    #39 4.221 Studio UI build reported success but dist/index.html is missing
    #39 ERROR: process "/bin/sh -c set -eu; ..." did not complete successfully: exit code: 1
    
    Under the old body that same state exited 0 and shipped.

Exit-path matrix (harness mirroring the RUN body):

case new old
success + index.html 0 0
success, index.html gone 1 0
timeout 124 retry ×3 → 124 0
SIGKILL 137 retry ×3 → 124 0
failure 1 / 2 1 / 2 0

Caveat on the happy-path build above: this host is arm64, so it ran natively and did not exercise the emulated path that broke 0.3.0. It proves the guard and the happy path, not that emulated arm64 succeeds. Bug 3 is what addresses the emulated path.

Bug 3, both target platforms built on an arm64 host:

  • before — buildkit spawns a full second chain, [linux/amd64 base 10/29][linux/amd64 base 21/29] RUN pnpm install --frozen-lockfile, which dies under emulation:

    #62 [linux/amd64 base 21/29] RUN pnpm install --frozen-lockfile
    #62 62.15 . postinstall: [safe-synthesizer] ✘ [ERROR] panic: runtime error: invalid memory address or nil pointer dereference
    #62 91.08 . postinstall: 💥 Some type generation failed.
    #62 ERROR: process "/bin/sh -c pnpm install --frozen-lockfile" did not complete successfully: exit code: 1
    

    A Go nil-pointer panic in orval SDK generation. The whole build exits 1. This host is arm64, so the emulated half is amd64 — the mirror image of CI, where an amd64 runner emulates arm64. Different symptom from the chunk-rendering hang, same root cause: the node toolchain does not survive QEMU.

  • after — only native arm64 stages execute (no linux/amd64 base/build at all), and the two exported trees are byte-identical:

    d6dd8e840ffa8b925e13644a0b1f7497df151b9d1bd8d19eb2648e9fd3b7c570  linux_amd64/artifacts
    d6dd8e840ffa8b925e13644a0b1f7497df151b9d1bd8d19eb2648e9fd3b7c570  linux_arm64/artifacts
    

    diff -r clean, 331 assets each, index.html 8710 B each.

Tests: pytest services/studio/tests/unit → 107 passed. New coverage for the container fallback, configured-path precedence, the /static/studio constant, both error-page branches, and the env-over-YAML ordering. ruff check / ruff format / ty check clean.

Note for reviewers

The config-reference-docs pre-commit hook cannot run from a git worktree — uv fails to fetch the nooa git dependency (fatal: not a git repository) before it reaches doc generation. I skipped it and hand-edited docs/set-up/config-reference.mdx; the line is byte-identical to what the generator emits for that field.

Summary by CodeRabbit

  • Bug Fixes

    • Improved Studio build validation to detect missing output files and report failures accurately.
    • Enhanced static asset discovery across packaged installations, container images, and source checkouts.
    • Added clearer recovery guidance when Studio assets are unavailable.
    • Preserved configuration precedence for explicitly configured asset paths and environment settings.
  • Documentation

    • Updated configuration guidance to explain static asset fallback locations and setup options.

The nmp-api image copies the Studio bundle to /static/studio, but the
packaged nmp/studio/static dir is only populated by the wheel build, so
any non-Helm run of the image (docker run, compose, quickstart) fell
through to the "assets are not built" page. Helm masked this by setting
studio.static_files_path explicitly.

Add /static/studio to the _get_static_files_path fallback chain rather
than baking NMP_STUDIO_STATIC_FILES_PATH into the Dockerfile: ServiceConfig
derives from EnvironmentFirstSettings, which orders env_settings ahead of
init_settings, so the env var would silently override an operator's
studio.static_files_path. The fallback also covers future images that lay
the bundle down without setting the var.

Also split the error page's recovery block. The nvm / make bootstrap-studio
instructions only apply to a source checkout; packaged installs now get the
NMP_STUDIO_STATIC_FILES_PATH knob and the container bundle location instead.

Signed-off-by: mschwab <mschwab@nvidia.com>
…ist [ASTD-354]

`status="$?"` sat after `fi`, so it captured the exit status of the `if`
compound command (always 0 when the condition fails and there is no else)
rather than the build's. Every studio build failure therefore fell into
`[ 0 != 124 ] && [ 0 != 137 ]` and ran `exit 0`. The RUN succeeded, the
three-attempt retry never once fired, and whatever was in dist/ shipped.

Vite copies publicDir into outDir from the `vite:prepare-out-dir` renderStart
hook, before chunks are rendered, so a build killed during chunk rendering
leaves dist/ holding only public/ files. That is exactly what the arm64
nmp-api:0.3.0 image carries at /static/studio: favicon.svg, sample-agents/,
sample-datasets/, no index.html, no assets/. The amd64 variant of the same
tag is fine, which matches the QEMU hang the existing comment describes.

Capture the status with `|| status="$?"` outside the `if`, and assert
dist/index.html exists before reporting success so a silently truncated
bundle fails the build rather than reaching an image.

Signed-off-by: mschwab <mschwab@nvidia.com>
@marcusds
marcusds requested review from a team as code owners August 4, 2026 18:34
@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The Studio UI Docker build now verifies dist/index.html. Studio asset lookup now includes /static/studio, with installation-specific recovery guidance and updated documentation and tests.

Changes

Studio asset delivery

Layer / File(s) Summary
Validate the Studio UI build
docker/base/Dockerfile.nmp-studio-ui
The build records the command status directly and fails when dist/index.html is missing after a successful build.
Resolve Studio assets and recovery guidance
services/studio/src/nmp/studio/service.py, services/studio/src/nmp/studio/config.py, services/studio/tests/unit/test_service.py, docs/set-up/config-reference.mdx
Asset lookup checks configured, packaged, container, and source-checkout locations. Missing-asset responses select source-build or packaged-install instructions. Tests and configuration documentation cover the updated behavior.

Suggested reviewers: nakolean

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main changes: serving container-baked Studio assets and preventing partial Studio bundles.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch astd-354-studio-assets-container/mschwab

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added the fix label Aug 4, 2026
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Signed-off-by: mschwab <mschwab@nvidia.com>
@github-actions

github-actions Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 30392/38416 79.1% 63.8%
Integration Tests 17995/37085 48.5% 21.0%

nmp-studio-ui declares platforms = [linux/amd64, linux/arm64], so buildkit
ran the whole node + pnpm + vite chain twice and emulated whichever half did
not match the builder. dist/ is architecture-independent JS/CSS/HTML, so the
second chain bought nothing and supplied the failure mode: emulated arm64
hangs in Vite chunk rendering, which is how nmp-api:0.3.0 shipped an arm64
bundle holding only public/ files.

Pin the node stages to $BUILDPLATFORM. The scratch artifacts stage is still
stamped per target platform, so nmp-api consumes it unchanged.

Verified by building both target platforms on an arm64 host. Before: a full
linux/amd64 base chain runs under emulation and dies in `pnpm install` with
a Go nil-pointer panic in orval SDK generation, failing the build. After:
only native arm64 stages run, and the two exported trees are byte-identical
(sha256 d6dd8e84..., 331 assets, index.html 8710 B each).

Signed-off-by: mschwab <mschwab@nvidia.com>

@svvarom svvarom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dockerfile LGTM

@marcusds
marcusds added this pull request to the merge queue Aug 4, 2026
Merged via the queue into main with commit 2db4cfc Aug 4, 2026
56 checks passed
@marcusds
marcusds deleted the astd-354-studio-assets-container/mschwab branch August 4, 2026 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants