Problem Description
Every wheel SDK install pulls in the ROCm compiler and development toolchain, whether or not the user will ever build anything. It roughly doubles the download and cannot be turned off.
therock_pip_package_specs (apps/rocm/src/therock.rs:991-998) hardcodes:
rocm[libraries,devel]==V torch==V torchvision==V torchaudio==V
The rocm meta-package's two extras are disjoint — libraries brings the runtime libraries, devel brings headers, static libraries, hipcc, and the full LLVM toolchain — so dropping devel cannot remove anything needed to run a model.
Measured from the published wheel sizes (compressed bytes, release channel, ROCm 7.10.0):
| target |
[libraries] |
[libraries,devel] |
saved |
| gfx110X-dgpu, Linux |
0.56 GiB |
2.04 GiB |
1.48 GiB (72%) |
| gfx110X-dgpu, Windows |
0.78 GiB |
1.97 GiB |
1.19 GiB (60%) |
| gfx94X-dcgpu, Linux |
1.70 GiB |
3.06 GiB |
1.36 GiB (44%) |
| gfx1151, Linux |
0.47 GiB |
1.81 GiB |
1.34 GiB (74%) |
The unpacked cost is larger still; only compressed sizes were measured.
Nothing in this repository needs devel at runtime. The SDK probe already treats it as optional — apps/rocm/src/therock.rs:2382-2390 wraps the _devel import in try/except and backfills the root and bin paths from the runtime packages, and validate_rocm_sdk_runtime_probe (apps/rocm/src/therock.rs:2283-2315) only requires the import to succeed, the paths to exist, and the runtime libraries to resolve. There is already a passing test for exactly this: runtime_only_rocm_sdk_probe_validates_without_devel_root.
No codepath compiles anything against ROCm headers: the vLLM install is a plain wheel install with no build isolation or source builds, lemonade downloads prebuilt backends, and ComfyUI has no custom-node build path. hipcc appears only as an existence marker for detecting a system ROCm install and in advice strings — it is never executed. cmake_path is written into the probe result and never read anywhere.
The one real reference is in the release packaging script, which prefers a different toolchain first and silently skips when none is found — a maintainer-only path, never hit by end users.
Suggested Fix
Make the toolchain opt-in with a --devel flag on rocm install sdk, defaulting to off:
/// Also install the ROCm compiler and headers, for building GPU code.
#[arg(long)]
devel: bool,
A positive bare boolean matches the dominant convention for this command's flags (--reinstall, --replace, --dkms); the codebase has exactly one negative flag, so --no-devel would match the outlier rather than the norm.
Three places echo the package spec in user-facing text and must stay in sync with the change: the package policy line, the progress label (which currently hardcodes "devel SDK"), and the resolution error.
If a behaviour-preserving landing is preferred, this could ship as an opt-out first and flip the default later — though given the evidence that nothing needs it, that mostly costs a second flag rename.
Verification
The runtime-only probe path is already unit-tested. What is not yet verified is a real end-to-end install without the extra — in particular whether the resolver pulls rocm-sdk-devel back in transitively via the torch wheels. That needs a GPU lane with a real SDK install; the existing GPU end-to-end lanes are the natural home, and a nightly-tagged scenario would avoid adding a cold multi-gigabyte download to the pull-request path.
One runtime behaviour worth confirming on hardware rather than assuming: Triton compiles kernels at run time under vLLM. On ROCm it bundles its own toolchain and uses the runtime packages, and nothing here points it at the devel tree — but a runtime-only vLLM serve run should confirm that.
Additional Information
Found while investigating unbounded disk growth. Sizes were read from the published index; no full install was performed.
Problem Description
Every wheel SDK install pulls in the ROCm compiler and development toolchain, whether or not the user will ever build anything. It roughly doubles the download and cannot be turned off.
therock_pip_package_specs(apps/rocm/src/therock.rs:991-998) hardcodes:The
rocmmeta-package's two extras are disjoint —librariesbrings the runtime libraries,develbrings headers, static libraries,hipcc, and the full LLVM toolchain — so droppingdevelcannot remove anything needed to run a model.Measured from the published wheel sizes (compressed bytes, release channel, ROCm 7.10.0):
[libraries][libraries,devel]The unpacked cost is larger still; only compressed sizes were measured.
Nothing in this repository needs
develat runtime. The SDK probe already treats it as optional —apps/rocm/src/therock.rs:2382-2390wraps the_develimport in try/except and backfills the root and bin paths from the runtime packages, andvalidate_rocm_sdk_runtime_probe(apps/rocm/src/therock.rs:2283-2315) only requires the import to succeed, the paths to exist, and the runtime libraries to resolve. There is already a passing test for exactly this:runtime_only_rocm_sdk_probe_validates_without_devel_root.No codepath compiles anything against ROCm headers: the vLLM install is a plain wheel install with no build isolation or source builds, lemonade downloads prebuilt backends, and ComfyUI has no custom-node build path.
hipccappears only as an existence marker for detecting a system ROCm install and in advice strings — it is never executed.cmake_pathis written into the probe result and never read anywhere.The one real reference is in the release packaging script, which prefers a different toolchain first and silently skips when none is found — a maintainer-only path, never hit by end users.
Suggested Fix
Make the toolchain opt-in with a
--develflag onrocm install sdk, defaulting to off:A positive bare boolean matches the dominant convention for this command's flags (
--reinstall,--replace,--dkms); the codebase has exactly one negative flag, so--no-develwould match the outlier rather than the norm.Three places echo the package spec in user-facing text and must stay in sync with the change: the package policy line, the progress label (which currently hardcodes "devel SDK"), and the resolution error.
If a behaviour-preserving landing is preferred, this could ship as an opt-out first and flip the default later — though given the evidence that nothing needs it, that mostly costs a second flag rename.
Verification
The runtime-only probe path is already unit-tested. What is not yet verified is a real end-to-end install without the extra — in particular whether the resolver pulls
rocm-sdk-develback in transitively via the torch wheels. That needs a GPU lane with a real SDK install; the existing GPU end-to-end lanes are the natural home, and a nightly-tagged scenario would avoid adding a cold multi-gigabyte download to the pull-request path.One runtime behaviour worth confirming on hardware rather than assuming: Triton compiles kernels at run time under vLLM. On ROCm it bundles its own toolchain and uses the runtime packages, and nothing here points it at the
develtree — but a runtime-only vLLM serve run should confirm that.Additional Information
Found while investigating unbounded disk growth. Sizes were read from the published index; no full install was performed.