fix(openai): forward request params from with_azure for parity with L…#6211
fix(openai): forward request params from with_azure for parity with L…#6211RaidLZ wants to merge 2 commits into
Conversation
…LM.__init__ LLM.with_azure() only forwarded a subset of the request-tuning parameters accepted by LLM.__init__, silently dropping store, metadata, prompt_cache_retention, extra_body, extra_headers and extra_query. Azure users therefore had no way to set them - e.g. extra_body is needed to pass Azure OpenAI "On Your Data" data_sources. Forward these params from with_azure into the LLM constructor, mirroring __init__. All default to NOT_GIVEN, so behaviour is unchanged unless explicitly set. Add hermetic unit tests covering both the forwarding and the NOT_GIVEN defaults.
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
store was saved on _opts but the chat() method never added it to the request kwargs, so it was silently dropped (the Responses API path forwards it correctly). Exposing store on with_azure surfaced this. Forward it alongside the other _opts fields and add regression tests asserting it reaches the request when set and is absent when unset.
There was a problem hiding this comment.
🚩 Pre-existing: user-provided extra_headers dict is mutated in-place across chat() calls
In livekit-agents/livekit/agents/inference/llm.py:412-413, the _run() method does extra_headers = self._extra_kwargs.setdefault("extra_headers", {}) followed by extra_headers.update(get_inference_headers()). Since chat() at livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/llm.py:979-980 assigns extra["extra_headers"] = self._opts.extra_headers (a direct reference, not a copy), repeated chat() calls will accumulate inference headers into the user's original dict object stored in _opts. This is a pre-existing issue (not introduced by this PR) but is now more likely to be encountered since with_azure() users can now pass extra_headers.
(Refers to lines 979-980)
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
LLM.with_azure() only forwarded a subset of the request-tuning parameters accepted by LLM.init, silently dropping store, metadata, prompt_cache_retention, extra_body, extra_headers and extra_query. Azure users therefore had no way to set them - e.g. extra_body is needed to pass Azure OpenAI "On Your Data" data_sources.
Forward these params from with_azure into the LLM constructor, mirroring init. All default to NOT_GIVEN, so behaviour is unchanged unless explicitly set.