Skip to content

fix(server): use GTT memory for Strix Halo VRAM monitoring#9

Open
jane-alesi wants to merge 1 commit into
mainfrom
fix/amd-unified-vram
Open

fix(server): use GTT memory for Strix Halo VRAM monitoring#9
jane-alesi wants to merge 1 commit into
mainfrom
fix/amd-unified-vram

Conversation

@jane-alesi

Copy link
Copy Markdown

What

Fix false VRAM pressure readings on AMD Strix Halo (gfx1105) and Strix Point (gfx1102) systems.

The VRAM monitor only reads mem_info_vram_* (2GB LPDDR5 chip) and ignores mem_info_gtt_* (124GB GTT). On Strix Halo, models run entirely in GTT — the LPDDR5 is a tiny buffer.

Symptoms Before Fix

  • 600+ log lines/min from nvidia-smi subprocess spawning
  • "VRAM pressure critical (99.0% >= 90.0%)" every 2 seconds
  • False eviction triggers when LPDDR5 has transient load spikes
  • Models evicted incorrectly despite 0.13% actual memory pressure

Symptoms After Fix

  • 0 VRAM-related log lines/min (on AMD systems)
  • Correct memory pressure reading: 0.13% (was 99%)
  • No false evictions
  • nvidia-smi subprocess skipped entirely on AMD

Technical Details

Strix Halo uses a unified memory architecture. The kernel exposes two memory pools:

  • mem_info_vram_* — 2GB LPDDR5 chip (tiny buffer)
  • mem_info_gtt_* — 124GB GTT (where models actually live)

The fix detects unified-memory GPUs by checking KFD gfx_target_version (gfx1102, gfx1105) and computes pressure as (vram_used + gtt_used) / (vram_total + gtt_total).

Discrete AMD GPUs (7900 XTX, etc.) and NVIDIA GPUs use the existing path unchanged.

nvidia-smi skip: Uses PCI vendor ID scanning (/sys/bus/pci/devices/*/vendor == 0x10de) — more robust than KFD-only check since KFD may not be present on all AMD systems.

Files Changed

  • src/cpp/include/lemon/system_info.h — header-only has_unified_memory() inline function
  • src/cpp/include/lemon/system_info.his_strix_apu() declaration in SystemInfo class
  • src/cpp/server/system_info.cpp — GTT-aware VRAM calculation + PCI-based nvidia-smi skip
  • src/cpp/server/system_info.cppis_strix_apu() implementation (KFD gfx_target_version scan)
  • test/cpp/test_amd_vram.cpp — unit tests for has_unified_memory() and vram ratio calculation
  • CMakeLists.txt — test target for test_amd_vram

Related Issues

On AMD Strix Halo (gfx1105) and Strix Point (gfx1102) APUs, the GPU
uses unified GTT memory as its primary pool. The existing VRAM monitor
only reads mem_info_vram_* (2GB LPDDR5 buffer), causing false 99%
pressure readings and log spam (600+ lines/min).

This fix:
1. Detects unified memory GPUs via KFD gfx_target_version
2. On unified GPUs, computes (vram_used + gtt_used) / (vram_total + gtt_total)
3. On discrete GPUs, uses the existing vram_only path
4. Skips nvidia-smi subprocess spawn entirely on AMD (reduces log spam)

Tested on: Ryzen AI MAX+ 395 (gfx1105), 126GB unified memory.
Before: VRAM pressure 99%, 600 log lines/min
After:  VRAM pressure 0.13%, 0 VRAM log lines/min
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant