[api][plan][runtime] Separate prompt arguments from message extra_args in BaseChatModelSetup.chat()#698
Open
weiqingy wants to merge 2 commits into
Open
[api][plan][runtime] Separate prompt arguments from message extra_args in BaseChatModelSetup.chat()#698weiqingy wants to merge 2 commits into
weiqingy wants to merge 2 commits into
Conversation
…s in BaseChatModelSetup.chat() `BaseChatModelSetup.chat()` previously filled prompt templates by flattening every input message's `extra_args` into a single map. This conflated chat metadata with template variables. Introduce an explicit `arguments` parameter on `chat()` (Java + Python) and carry the same field on `ChatRequestEvent`, then thread it through `ChatModelAction` to the setup on both round 1 and tool-response continuations so multi-turn flows keep re-filling the template correctly. `ChatMessage.extra_args` is unchanged and still carries `externalId`, `STRUCTURED_OUTPUT`, OpenAI `refusal`, Ollama `reasoning`, and other provider-specific metadata used by chat-model connections. Closes apache#220
2 tasks
Collaborator
Author
|
Looks like the 1 failing check is unrelated to this PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #220
Purpose of change
BaseChatModelSetup.chat()(Java + Python) previously filled prompt templates by flattening every input message'sextra_argsinto a single map. This conflated chat-message metadata with prompt-template variables and forced callers to stuff template values into a generic metadata bag.This PR adds an explicit
argumentsparameter onchat()and carries the same field onChatRequestEvent.ChatModelActionextracts it from the event and forwards to the setup on both round 1 and tool-response continuations, so multi-turn flows keep re-filling the template correctly.ChatMessage.extra_argsis unchanged and still carriesexternalId,STRUCTURED_OUTPUT, OpenAIrefusal, Ollamareasoning, and other provider-specific metadata used by chat-model connections.Tests
chat(messages, arguments, parameters)formats thePrompttemplate using values fromarguments.ChatMessagewithextra_argsset no longer feeds the prompt template — proves the cutover.chat()invocations with the sameargumentsre-fill the template each time.processChatRequestOrToolResponsewith aToolResponseEventextracts the persistedargumentsfrom the saved tool-request context and forwards tochat_model.chat(...)on round 2 — locks the multi-turn contract.PythonChatModelSetupTest.testChatasserts"arguments"flows into Pemja kwargs marshalled to Python.test_built_in_actions.py,built_in_action_async_execution_test.py, ande2e_tests_mcp/mcp_test.pypreserve their outcome assertions post-cutover — proves behavior parity../tools/ut.sh -j;uv run pytest: 509 passed / 13 skipped)../tools/lint.sh -cclean.API
Yes — this is a breaking API change:
BaseChatModelSetup: the existing 2-argchat(List<ChatMessage>, Map<String, Object> parameters)overload is removed (it would erase to the same signature as a hypotheticalchat(messages, arguments)). The new primary form ischat(List<ChatMessage>, Map<String, Object> arguments, Map<String, Object> parameters). The 1-argchat(List<ChatMessage>)convenience overload is retained.BaseChatModelSetup.chat:arguments: Mapping[str, Any] | None = Noneis added betweenmessagesand**kwargs.ChatRequestEvent: new 4-arg constructor(String model, List<ChatMessage> messages, @Nullable Map<String, Object> arguments, @Nullable Object outputSchema). Existing 2-arg and 3-arg legacy constructors continue to work (delegating with empty arguments).ChatRequestEvent.__init__:arguments: Dict[str, Any] | None = Noneadded betweenmessagesandoutput_schema.Migration: callers that previously set template variables via
ChatMessage.extra_argsshould move them toChatRequestEvent.arguments. All in-repo callers (3 Java examples, 3 Python examples, 2 e2e tests, 1 runtime test) are migrated in this PR.Documentation
doc-neededdoc-not-neededdoc-included