Skip to content

server : properly handle null llama_context#25868

Open
fairydreaming wants to merge 1 commit into
ggml-org:masterfrom
fairydreaming:server-null-ctx-check
Open

server : properly handle null llama_context#25868
fairydreaming wants to merge 1 commit into
ggml-org:masterfrom
fairydreaming:server-null-ctx-check

Conversation

@fairydreaming

Copy link
Copy Markdown
Collaborator

Overview

When researching #25851 I noticed that llama-server does not handle case where llama_context creation failed and ctx_tgt is null, instead it crashes trying to access null context pointer. This PR adds a context nullity check to handle this case.

Additional information

This is purely cosmetic improvement - instead of:

./bin/llama-server -m ~/ggufs/DeepSeek-V4-Flash.gguf -c 262144 -ctk q8_0 -ctv q8_0 -lv 4 -fit off -fa off -cmoe
...
0.10.162.893 I load_tensors: offloading output layer to GPU
0.10.162.901 I load_tensors: offloading 42 repeating layers to GPU
0.10.162.902 I load_tensors: offloaded 44/44 layers to GPU
0.10.162.917 I load_tensors:   CPU_Mapped model buffer size = 141362.00 MiB
0.10.162.920 I load_tensors:        CUDA0 model buffer size =  7766.93 MiB
0.10.683.593 I cmn  common_init_: added <|end▁of▁sentence|> logit bias = -inf
0.10.692.115 E llama_init_from_model: V cache quantization requires flash_attn
0.10.692.118 E cmn  common_init_: failed to create context with model '/home/phm/ggufs/DeepSeek-V4-Flash.gguf'
0.10.692.156 Segmentation fault (core dumped)

we will have:

./bin/llama-server -m ~/ggufs/DeepSeek-V4-Flash.gguf -c 262144 -ctk q8_0 -ctv q8_0 -lv 4 -fit off -fa off -cmoe
...
0.09.491.879 I load_tensors: offloading output layer to GPU
0.09.491.885 I load_tensors: offloading 42 repeating layers to GPU
0.09.491.885 I load_tensors: offloaded 44/44 layers to GPU
0.09.491.892 I load_tensors:   CPU_Mapped model buffer size = 141362.00 MiB
0.09.491.893 I load_tensors:        CUDA0 model buffer size =  7766.93 MiB
0.10.004.591 I cmn  common_init_: added <|end▁of▁sentence|> logit bias = -inf
0.10.005.510 E llama_init_from_model: V cache quantization requires flash_attn
0.10.005.512 E cmn  common_init_: failed to create context with model '/home/phm/ggufs/DeepSeek-V4-Flash.gguf'
0.10.005.515 E cmn  common_init_: failed to create context with model '/home/phm/ggufs/DeepSeek-V4-Flash.gguf'
0.10.005.520 E srv    load_model: failed to create_context with model '/home/phm/ggufs/DeepSeek-V4-Flash.gguf'
0.10.005.522 I srv    operator(): operator(): cleaning up before exit...
0.10.016.392 E srv  llama_server: exiting due to model loading error

Requirements

@fairydreaming
fairydreaming requested a review from a team as a code owner July 18, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants