fix(backends): darwin/metal support across purego Go backends#10481
Open
localai-bot wants to merge 2 commits into
Open
fix(backends): darwin/metal support across purego Go backends#10481localai-bot wants to merge 2 commits into
localai-bot wants to merge 2 commits into
Conversation
The parakeet-cpp backend had no macOS support and panicked at startup on Apple/Metal nodes when purego.Dlopen could not find "libparakeet.so". Fix it across the same four layers the sibling voxtral backend already handles correctly: - main.go: default the dlopen target to libparakeet.dylib on darwin (runtime.GOOS), libparakeet.so elsewhere; PARAKEET_LIBRARY still wins. - Makefile: also stage the built libparakeet.dylib next to the Go sources. - package.sh: accept either the Linux .so[.X.Y] or the macOS .dylib when bundling instead of hard-failing when no .so is present (the macOS case); note that on Darwin only system frameworks are linked. - run.sh: on Darwin set DYLD_LIBRARY_PATH and PARAKEET_LIBRARY to the packaged .dylib; keep LD_LIBRARY_PATH + .so on Linux. Mirrors backend/go/voxtral. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
7159582 to
9f7c58d
Compare
The parakeet-cpp fix in the previous commit was an instance of a bug shared by nearly every purego/dlopen Go backend: the dlopen target was hardcoded to a .so name and run.sh exported only LD_LIBRARY_PATH, so the backend panicked at startup on macOS/Apple-Metal nodes (dyld needs the .dylib name and DYLD_LIBRARY_PATH). voxtral was the only backend handling this correctly. Apply the same four-layer fix (mirroring backend/go/voxtral) to the remaining affected backends: whisper, sherpa-onnx, ced, stablediffusion-ggml, vibevoice-cpp, qwen3-tts-cpp, omnivoice-cpp, crispasr, acestep-cpp, locate-anything-cpp, depth-anything-cpp, rfdetr-cpp, sam3-cpp, localvqe Per backend: - main.go (sherpa-onnx: backend.go, two libraries): default the dlopen target to the .dylib on darwin (runtime.GOOS), .so elsewhere; the existing <BACKEND>_LIBRARY env override still wins. - run.sh: on Darwin set DYLD_LIBRARY_PATH and point <BACKEND>_LIBRARY at the packaged .dylib; keep LD_LIBRARY_PATH + the Linux CPU-variant (avx/avx2/avx512) selection unchanged in the else branch. - package.sh: also bundle the .dylib and stop hard-failing when no .so is present (the macOS case). - Makefile: also stage the built .dylib. Notes: - stablediffusion-ggml and acestep-cpp build their lib as a CMake MODULE, which emits .so (not .dylib) on macOS; run.sh prefers .dylib and falls back to .so so both layouts work. - sherpa-onnx was already partly darwin-aware (Makefile/package.sh); only run.sh and the two dlopen defaults needed fixing. Linux behavior is unchanged. Verified gofmt-clean and `CGO_ENABLED=0 go build` for every backend. Assisted-by: Claude:claude-opus-4-8 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
purego/dlopen-based Go backends crash at startup on macOS / Apple-Metal nodes. The dlopen target is hardcoded to a
.soname andrun.shexports onlyLD_LIBRARY_PATH(which macOS dyld ignores), so the native library is never found:This was observed in production (parakeet-cpp STT dying on a Mac mini M4 worker node). Auditing the tree showed it is a shared bug class:
backend/go/voxtralwas the only purego backend handling darwin correctly. Every other one was affected.Fix
Apply the four-layer
voxtralpattern to all affected backends:backend.go) — default the dlopen target to the.dylibondarwinviaruntime.GOOS,.soelsewhere. The existing<BACKEND>_LIBRARYenv override still wins.DYLD_LIBRARY_PATHand point<BACKEND>_LIBRARYat the packaged.dylib; keepLD_LIBRARY_PATHand the Linux CPU-variant (avx/avx2/avx512) selection unchanged in theelsebranch..dylib, and stop hard-failing (set -e/exit 1) when no.sois present (the macOS case)..dylib.Backends fixed (15)
parakeet-cpp(first commit), then:whisper,sherpa-onnx,ced,stablediffusion-ggml,vibevoice-cpp,qwen3-tts-cpp,omnivoice-cpp,crispasr,acestep-cpp,locate-anything-cpp,depth-anything-cpp,rfdetr-cpp,sam3-cpp,localvqe.voxtralwas already correct and is the reference.opusis pure-Go (no dlopen) and is unaffected.Notes
stablediffusion-ggmlandacestep-cppbuild their library as a CMake MODULE, which emits.so(not.dylib) on macOS. Theirrun.shprefers.dyliband falls back to.so, so both layouts work.sherpa-onnxwas already partly darwin-aware (Makefile/package.sh) and loads two libraries; only itsrun.shand the two dlopen defaults needed fixing.elsebranch;.sostaging still wins first).Tested
PARAKEET_LIBRARYat the builtlibparakeet.dylibmakes the backend start cleanly ([parakeet-cpp] ABI=5, gRPC listening, no panic). The dylib links only system frameworks (libSystem, Accelerate, libc++), so nothing extra needs bundling on darwin (same as voxtral's Darwin note).gofmt -lclean andCGO_ENABLED=0 go build ./backend/go/<name>/succeeds for all 15 backends;bash -nclean on every changedrun.sh/package.sh.Assisted-by: Claude:claude-opus-4-8