Skip to content

Reasoning is not displayed with the DMR provider (and reasoning model) #2262

@k33g

Description

@k33g

Reasoning is not displayed with the DMR provider when using a reasoning model.

I did the tests with huggingface.co/unsloth/nemotron-3-nano-30b-a3b-gguf:Q3_K_M. It's a reasoning model.

I tried with and without /think (as a prefix of the user prompt)

I tried to use: thinking_budget, and even some runtime flags: --verbose-prompt","--special", but the reasoning is not displayed.

Model configuration:

models:

  nemotron3:
    provider: dmr
    model: huggingface.co/unsloth/nemotron-3-nano-30b-a3b-gguf:Q3_K_M
    thinking_budget: medium
    temperature: 0.0
    top_p: 0.8
    presence_penalty: 1.5
    max_tokens: 32768

    provider_opts:
      runtime_flags: ["--context_size=65536","--verbose-prompt","--special"]
      top_k: 20
      repetition_penalty: 1.05

I did the test with Docker Desktop UI in the models panel, and it works:

Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions