Skip to content

fix(openvino): explicit ov::Tensor frees in ggml_backend_openvino_free#132

Merged
ravi9 merged 1 commit intoravi9:ov-update-1from
thedanhoffman:ov_tensor_free
Apr 14, 2026
Merged

fix(openvino): explicit ov::Tensor frees in ggml_backend_openvino_free#132
ravi9 merged 1 commit intoravi9:ov-update-1from
thedanhoffman:ov_tensor_free

Conversation

@thedanhoffman
Copy link
Copy Markdown

Overview

The following throws a segmentation fault on exit.

export GGML_OPENVINO_DEVICE=NPU
./bin/llama-cli -m Llama-3.2-1B-Instruct-Q4_0.gguf -ngl 99 -np 1 -c 512 --simple-io --single-turn -n 1 -p hi

Root-caused to a use-after-free involving OpenVINO's ZeroMemPool. Goes like this

  1. ZeroMemPool::allocate_zero_memory() returns std::shared_ptr<ZeroMem>
  2. std::shared_ptr<ZeroMem> has a custom destructor method of the form [this](ZeroMem *ptr) { delete_pool_entry(ptr); } (where delete_pool_entry is a non-static member function of ZeroMemPool)
  3. this is invalid by the time the destructor runs because RAII does not clean up std::shared_ptr<ZeroMem> consistently

This is downstream of std::shared_ptr<void> being used for runtime_context not propagating the destructor call to class members.

Went ahead and defined a clear_caches method that does this destruction explicitly (but not using std::shared_ptr<void> seems like a better approach considering the failure mode here, will leave that to the maintainers unless they suggest I change that here).

Additional information

This PR (and all other PRs I'll file in this space) use a recent from-source build of OpenVINO.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: I threw an AI at this, but I have a solid understanding of the mechanics at play and the fix is targeted enough to not cause collateral issues.

Copy link
Copy Markdown
Collaborator

@cavusmustafa cavusmustafa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ravi9 ravi9 merged commit 3f6d5fd into ravi9:ov-update-1 Apr 14, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants