Skip to content

fix(backends): darwin/metal support across purego Go backends#10481

Open
localai-bot wants to merge 2 commits into
masterfrom
fix/parakeet-cpp-darwin-dylib
Open

fix(backends): darwin/metal support across purego Go backends#10481
localai-bot wants to merge 2 commits into
masterfrom
fix/parakeet-cpp-darwin-dylib

Conversation

@localai-bot

@localai-bot localai-bot commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Problem

purego/dlopen-based Go backends crash at startup on macOS / Apple-Metal nodes. The dlopen target is hardcoded to a .so name and run.sh exports only LD_LIBRARY_PATH (which macOS dyld ignores), so the native library is never found:

panic: parakeet-cpp: dlopen "libparakeet.so": dlopen(libparakeet.so, 0x000A): tried:
  'libparakeet.so' (no such file), '/usr/lib/libparakeet.so' (no such file, not in dyld cache),
  '/usr/local/lib/libparakeet.so' (no such file, not in dyld cache)

This was observed in production (parakeet-cpp STT dying on a Mac mini M4 worker node). Auditing the tree showed it is a shared bug class: backend/go/voxtral was the only purego backend handling darwin correctly. Every other one was affected.

Fix

Apply the four-layer voxtral pattern to all affected backends:

  • main.go (sherpa-onnx: backend.go) — default the dlopen target to the .dylib on darwin via runtime.GOOS, .so elsewhere. The existing <BACKEND>_LIBRARY env override still wins.
  • run.sh — on Darwin export DYLD_LIBRARY_PATH and point <BACKEND>_LIBRARY at the packaged .dylib; keep LD_LIBRARY_PATH and the Linux CPU-variant (avx/avx2/avx512) selection unchanged in the else branch.
  • package.sh — also bundle the .dylib, and stop hard-failing (set -e / exit 1) when no .so is present (the macOS case).
  • Makefile — also stage the built .dylib.

Backends fixed (15)

parakeet-cpp (first commit), then: whisper, sherpa-onnx, ced, stablediffusion-ggml, vibevoice-cpp, qwen3-tts-cpp, omnivoice-cpp, crispasr, acestep-cpp, locate-anything-cpp, depth-anything-cpp, rfdetr-cpp, sam3-cpp, localvqe.

voxtral was already correct and is the reference. opus is pure-Go (no dlopen) and is unaffected.

Notes

  • stablediffusion-ggml and acestep-cpp build their library as a CMake MODULE, which emits .so (not .dylib) on macOS. Their run.sh prefers .dylib and falls back to .so, so both layouts work.
  • sherpa-onnx was already partly darwin-aware (Makefile/package.sh) and loads two libraries; only its run.sh and the two dlopen defaults needed fixing.
  • Linux behavior is byte-for-byte unchanged (the Linux paths stay in the else branch; .so staging still wins first).

Tested

  • Verified on an M4 Mac that pointing PARAKEET_LIBRARY at the built libparakeet.dylib makes the backend start cleanly ([parakeet-cpp] ABI=5, gRPC listening, no panic). The dylib links only system frameworks (libSystem, Accelerate, libc++), so nothing extra needs bundling on darwin (same as voxtral's Darwin note).
  • gofmt -l clean and CGO_ENABLED=0 go build ./backend/go/<name>/ succeeds for all 15 backends; bash -n clean on every changed run.sh/package.sh.

Assisted-by: Claude:claude-opus-4-8

The parakeet-cpp backend had no macOS support and panicked at startup on
Apple/Metal nodes when purego.Dlopen could not find "libparakeet.so".
Fix it across the same four layers the sibling voxtral backend already
handles correctly:

- main.go: default the dlopen target to libparakeet.dylib on darwin
  (runtime.GOOS), libparakeet.so elsewhere; PARAKEET_LIBRARY still wins.
- Makefile: also stage the built libparakeet.dylib next to the Go sources.
- package.sh: accept either the Linux .so[.X.Y] or the macOS .dylib when
  bundling instead of hard-failing when no .so is present (the macOS case);
  note that on Darwin only system frameworks are linked.
- run.sh: on Darwin set DYLD_LIBRARY_PATH and PARAKEET_LIBRARY to the
  packaged .dylib; keep LD_LIBRARY_PATH + .so on Linux.

Mirrors backend/go/voxtral.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler mudler force-pushed the fix/parakeet-cpp-darwin-dylib branch from 7159582 to 9f7c58d Compare June 24, 2026 16:03
The parakeet-cpp fix in the previous commit was an instance of a bug
shared by nearly every purego/dlopen Go backend: the dlopen target was
hardcoded to a .so name and run.sh exported only LD_LIBRARY_PATH, so the
backend panicked at startup on macOS/Apple-Metal nodes (dyld needs the
.dylib name and DYLD_LIBRARY_PATH). voxtral was the only backend handling
this correctly.

Apply the same four-layer fix (mirroring backend/go/voxtral) to the
remaining affected backends:

  whisper, sherpa-onnx, ced, stablediffusion-ggml, vibevoice-cpp,
  qwen3-tts-cpp, omnivoice-cpp, crispasr, acestep-cpp, locate-anything-cpp,
  depth-anything-cpp, rfdetr-cpp, sam3-cpp, localvqe

Per backend:
- main.go (sherpa-onnx: backend.go, two libraries): default the dlopen
  target to the .dylib on darwin (runtime.GOOS), .so elsewhere; the
  existing <BACKEND>_LIBRARY env override still wins.
- run.sh: on Darwin set DYLD_LIBRARY_PATH and point <BACKEND>_LIBRARY at
  the packaged .dylib; keep LD_LIBRARY_PATH + the Linux CPU-variant
  (avx/avx2/avx512) selection unchanged in the else branch.
- package.sh: also bundle the .dylib and stop hard-failing when no .so is
  present (the macOS case).
- Makefile: also stage the built .dylib.

Notes:
- stablediffusion-ggml and acestep-cpp build their lib as a CMake MODULE,
  which emits .so (not .dylib) on macOS; run.sh prefers .dylib and falls
  back to .so so both layouts work.
- sherpa-onnx was already partly darwin-aware (Makefile/package.sh); only
  run.sh and the two dlopen defaults needed fixing.

Linux behavior is unchanged. Verified gofmt-clean and
`CGO_ENABLED=0 go build` for every backend.

Assisted-by: Claude:claude-opus-4-8
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@localai-bot localai-bot changed the title fix(parakeet-cpp): darwin/metal support (libparakeet.dylib + DYLD path) fix(backends): darwin/metal support across purego Go backends Jun 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants