feat: tool types + registry + ToolHandler trait (PR A)#80
Conversation
maralbahari
left a comment
There was a problem hiding this comment.
@ashwing thank you. overall tools/ module looks well-structured. I have added some inline comments and it would be great to make sure the serde shapes and normalization logic are validated against real wire payloads before we merge.
We already have multi-turn tool call cassettes at crates/agentic-core/tests/cassettes/tool_calls/multi_turn/ responses_tool_calls_3turn.yaml, responses_tool_calls_5turn.yaml, and responses_tool_calls_parallel.yaml. Each turn's request.body.tools is the exact JSON the gateway receives, so parsing it into Vec<ResponsesTool> is the same validation as a real request without needing any running server.
The existing accumulator_cassette_test.rs already has the load_turn_cassette_from helper and Turn struct that deserializes request bodies as serde_json::Value. you can reuse that infrastructure directly in a new tool_normalization_test.rs.
Specifically the tests that cover:
serde_json::from_value::<Vec<ResponsesTool>>(request.body["tools"]) parses without error for every turn across all three cassettes
Each parsed tool normalizes correctly via to_function_tool(). The Function variants produce a FunctionTool with the right name/type, non-function variants return None
ToolRegistry::build on the cassette tools produces the expected entries and lookup by name works
Full round-trip: deserialize request.body into RequestPayload, call to_upstream_request(), assert the upstream tools field only contains FunctionTool entries and none of the gateway-only types leak through
Regarding MCP related code like DiscoveredTool, the discover() method, and the MCP branch in ToolRegistry::build, please don't add stub functions we can't test. discover() always returns Ok(vec![]) and DiscoveredTool is never populated, so this is dead code we can't validate.
since MCP is not in scope for this PR, please remove DiscoveredTool, the discover() method from ToolHandler, and the discovered parameter from normalize(). We can add MCP support once we have a way to test it.
1. Move tool behaviors to src/tool/ — types/ is for wire shapes only (registry, handler trait, normalization, FunctionHandler now live in src/tool/; types/tools/ keeps only ResponsesTool + param structs) 2. Replace function_tool_from_parts(&Value) with From<&FunctionToolParam> for FunctionTool — typed conversion over erased JSON (api-from-not-into) 3. Remove dead MCP stubs — DiscoveredTool, discover() on ToolHandler, and the discovered param from normalize() removed; untestable without a real MCP server, will be added when MCP handler lands in PR C 4. Strip section dividers (// ----) from all files per style feedback 5. Add cassette-based tests in tool_normalization_test.rs — validates Vec<ResponsesTool> parsing, to_function_tool() normalization, ToolRegistry::build lookup, and full RequestPayload round-trip against real wire payloads from the openai 3turn/5turn/parallel cassettes (12 new tests, all passing) Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Rebased on main after vllm-project#79 and vllm-project#81 merged. Adds src/tool/ — the behavioral layer that complements the wire types already in types/tools/ (merged via PR vllm-project#79): - tool/handler.rs — ToolHandler trait, ToolOutput, ToolError - tool/registry.rs — ToolType, ToolEntry, ToolRegistry::build/lookup/etc - tool/function.rs — FunctionHandler + From<&FunctionToolParam> for FunctionTool - tool/normalize.rs — ResponsesTool::to_function_tool(), From<ToolOutput> Also adds types/tools/ wire types (params.rs with ResponsesTool enum, param structs, NonEmptyToolName), EmptyToolNameError, and wires normalize into RequestPayload::to_upstream_request() so vLLM always receives Vec<FunctionTool>. 12 cassette-based tests in tool_normalization_test.rs validate the full pipeline against real multi-turn tool-call cassettes. Addresses all PR vllm-project#80 review feedback: - types/ → wire shapes only; tool/ → behaviors - From<&FunctionToolParam> for FunctionTool (typed conversion) - MCP registry entries deferred to PR C (discovery not yet wired) - EmptyToolNameError in types/ (no cross-layer import) - ToolOutput derives Debug + Clone Signed-off-by: Ashwin Giridharan <girida@amazon.com>
18338b2 to
590fede
Compare
|
Done across all commits on this rebase. Specifically: 1. Section dividers — stripped from all files. 2. 3. 4. Cassette tests + MCP stubs removed — added Also fixed two issues caught during review: Rebased on main after #79 and #81 — single clean commit, 264 tests pass. |
|
@ashwing thank you for addressing the issue. Trait shape:
The problem is that the trait forces every future tool handler to implement A cleaner split would be two traits: pub trait ToolHandler {
fn tool_type(&self) -> ToolType;
fn validate(&self, param: &Value) -> Result<(), ToolError>;
fn normalize(&self, param: &Value) -> Vec<FunctionTool>;
}
pub trait GatewayExecutor: ToolHandler {
fn execute(
&self,
tool_name: &str,
arguments: &str,
config: &Value,
) -> Pin<Box<dyn Future<Output = Result<ToolOutput, ToolError>> + Send + '_>>;
}
|
…n FunctionToolParam Addresses Maral's review comments: 1. Split ToolHandler into two traits (design: every method must apply to all implementors): - ToolHandler: tool_type/validate/normalize — all tool types implement this - GatewayExecutor: ToolHandler + 'static + execute() — gateway-only tools FunctionHandler now implements only ToolHandler, making it a compile-time error to call execute() on a client-owned tool. Add _assert_gateway_executor_dyn_compatible test to lock in Arc<dyn GatewayExecutor> object safety. 2. Remove #[allow(unreachable_patterns)] + _ catch-all from normalize.rs and registry.rs — compiler now enforces exhaustiveness when new ResponsesTool variants are added (Maral's inline comment). 3. FunctionToolParam.name: String -> NonEmptyToolName — serde rejects empty names at deserialization time, eliminating three divergent empty-name checks (registry build, to_function_tool, FunctionHandler validate). FunctionHandler::normalize now returns vec![] + warn on invalid param instead of panicking with .expect(). Signed-off-by: Ashwin Giridharan <girida@amazon.com>
|
@maralbahari The original design had That said, your point stands — Done. Split into:
Also applied |
maralbahari
left a comment
There was a problem hiding this comment.
@ashwing Thanks. LGTM
Implements the gateway-side agentic tool loop on top of the ToolRegistry and GatewayExecutor traits landed in PR A (vllm-project#80): - executor/dispatch.rs: LoopDecision enum (#[non_exhaustive]) + dispatch_tools() — classifies FunctionToolCall items via ToolRegistry::gateway_owned(), executes in parallel with 30s per-call timeout, maps failures to error-JSON FunctionCallOutput items (never aborts the loop on tool error). - executor/agentic_loop.rs: execute_loop() — multi-turn orchestrator that clears all three persistence triggers before looping and restores original IDs on the final payload. Rejects stream=true (StreamTee is a future PR). Hard guard of 128 iterations, soft cap via max_iterations param (default: 10). Client-owned function tools (ToolType::Function) return Done for now; RequiresAction and ContinuePartial are deferred per staging agreement in PR vllm-project#67 — LoopDecision is #[non_exhaustive] to make the addition safe. MCP tool names are absent from the registry until PR C adds discovery; any function_call for an MCP tool name is treated as client-owned. 244 tests pass; cargo clippy --workspace --all-targets -- -D warnings clean. Signed-off-by: Ashwin Giridharan <girida@amazon.com>
Summary
Implements PR A from the tool framework design:
ResponsesTool— replacespub type ResponsesTool = FunctionToolalias with a#[serde(tag = "type")]tagged enum covering all five Responses API tool types (function,mcp,web_search_preview,file_search,code_interpreter). Wire-compatible: existing{"type":"function",...}requests deserialize unchanged. Marked#[non_exhaustive]for future API additions.ToolRegistry— request-scopedHashMap<name → ToolEntry>built fromRequestPayload.tools+ any server-discovered tools. Providesgateway_owned/client_ownedsplit for post-inference dispatch routing (PR B).ToolHandlertrait —validate / discover / normalize / executelifecycle.Send + SyncforArc<dyn ToolHandler>. Defaultdiscover()is a no-op;execute()must be implemented per handler type.FunctionHandler— trivialToolHandlerimpl fortype: "function"tools. Normalizes toFunctionTool, never gateway-executes (client-owned).ResponsesTool::to_function_tool()wired intoRequestPayload::to_upstream_request(). vLLM always receivesVec<FunctionTool>, never rawResponsesToolvariants. Function tools pass through unchanged; gateway-managed types (mcp,web_search_preview,file_search,code_interpreter) are dropped with atracing::debug!until their handlers land in PR B/C.From<ToolOutput> for FunctionToolResultMessage— zero-friction conversion for PR B's dispatch loop.McpToolParamincludesheaders: Option<HashMap<String,String>>for per-server auth forwarding.What vLLM sees after this PR
{"type":"function","name":"foo",...}{"type":"function","name":"foo",...}✅ unchanged{"type":"mcp","server_label":"db",...}{"type":"web_search_preview"}Gateway-type tools are parsed, validated, and stored in
ResponseMetadata.effective_toolsfor rehydration across turns — they just don't reach vLLM until realnormalize()implementations land.No execution logic in this PR: no
dispatch_tools, noLoopDecision, no changes toengine.rs/rehydrate.rs/persist.rs.Depends on: #64 (io types refactor — merged). Unblocks: PR B (dispatch), PR C (MCP handler), CustomCodexTool (#68).
Test Plan
Key new tests in
types::tools::tests:ResponsesToolvariants round-trip through serdeToolRegistrylookup,gateway_owned/client_ownedsplit,allowed_toolsfiltering, duplicate-name warning, empty-name skipFunctionHandler::validate()rejects empty namesto_upstream_request()with mixed tools — only function tool reaches vLLMtools: null(empty array never sent to vLLM)