feat: Refactor Agents API to use FastAPI Router

[skamenan7]

Summary

This PR migrates the Agents API from the legacy @webmethod decorator pattern to the new FastAPI Router pattern, following the established Benchmarks API migration as a reference.

Changes

New Files

• src/llama_stack_api/agents/__init__.py - Module exports with docstring
• src/llama_stack_api/agents/api.py - Protocol definition
• src/llama_stack_api/agents/models.py - Pydantic request models
• src/llama_stack_api/agents/fastapi_routes.py - FastAPI router implementation

Modified Files

• src/llama_stack_api/__init__.py - Export request models from main package
• src/llama_stack/providers/inline/agents/meta_reference/agents.py - Updated to accept request model inputs
• src/llama_stack/core/server/fastapi_router_registry.py - Register agents router

Key Implementation Details

1. Request Models: All 5 API operations now use Pydantic request models:

   • CreateResponseRequest
   • RetrieveResponseRequest
   • ListResponsesRequest
   • ListResponseInputItemsRequest
   • DeleteResponseRequest

2. OpenAPI Spec: Properly documents POST /v1/responses with:

   • Status code 200 (not 204)
   • Both application/json and text/event-stream content types
   • Correct schema references

Testing

Test case: Create Response (Non-Streaming)

Request:

POST /v1/responses

{
  "model": "ollama/llama3.2:3b-instruct-fp16",
  "input": "Tell me a one-liner joke.",
  "stream": false
}

Response (200 OK):

{
  "id": "resp_ef4879f3-f12d-421a-84ac-d461c68e2767",
  "object": "response",
  "status": "completed",
  "output": [{
    "type": "message",
    "role": "assistant",
    "content": [{
      "type": "output_text",
      "text": "A man walked into a library and asked the librarian..."
    }]
  }],
  "usage": {"input_tokens": 33, "output_tokens": 54, "total_tokens": 87}
}

Test case: Create Response (Streaming)

Request:

POST /v1/responses

{
  "model": "ollama/llama3.2:3b-instruct-fp16",
  "input": "Count 1 to 3.",
  "stream": true
}

Response (SSE Stream):

data: {"type":"response.created",...}
data: {"type":"response.output_text.delta","delta":"1"}
data: {"type":"response.output_text.delta","delta":", "}
data: {"type":"response.output_text.delta","delta":"2"}
data: {"type":"response.output_text.delta","delta":", "}
data: {"type":"response.output_text.delta","delta":"3"}
data: {"type":"response.completed",...}

Test case: Tool Calling

Request:

POST /v1/responses

{
  "model": "ollama/llama3.2:3b-instruct-fp16",
  "input": "What is the weather in San Francisco?",
  "tools": [{
    "type": "function",
    "name": "get_weather",
    "parameters": {"type": "object", "properties": {"location": {"type": "string"}}}
  }]
}

Response (200 OK):

{
  "output": [{
    "type": "function_call",
    "name": "get_weather",
    "arguments": "{\"location\":\"San Francisco\"}"
  }]
}

Test case: Get Response

Request:

GET /v1/responses/resp_ef4879f3-f12d-421a-84ac-d461c68e2767

Response (200 OK):

{
  "id": "resp_ef4879f3-f12d-421a-84ac-d461c68e2767",
  "object": "response",
  "status": "completed",
  "output": [{
    "type": "message",
    "role": "assistant",
    "content": [{"type": "output_text", "text": "A man walked into a library..."}]
  }]
}

Test case: List Responses

Request:

GET /v1/responses

Response (200 OK):

{
  "data": [
    {"id": "resp_1cd72689-49f8-465e-9e3c-ac06ee3b3386", "object": "response", ...},
    {"id": "resp_dfe88aaa-9550-48e3-a4f4-dc189c78dc7c", "object": "response", ...},
    {"id": "resp_ef4879f3-f12d-421a-84ac-d461c68e2767", "object": "response", ...}
  ],
  "has_more": false,
  "object": "list"
}

Test case: List Input Items

Request:

GET /v1/responses/resp_ef4879f3-f12d-421a-84ac-d461c68e2767/input_items

Response (200 OK):

{
  "data": [{
    "type": "message",
    "role": "user",
    "content": [{"type": "input_text", "text": "Tell me a one-liner joke."}]
  }],
  "object": "list"
}

Test case: Delete Response

Request:

DELETE /v1/responses/resp_ef4879f3-f12d-421a-84ac-d461c68e2767

Response (200 OK):

{
  "id": "resp_ef4879f3-f12d-421a-84ac-d461c68e2767",
  "object": "response",
  "deleted": true
}

Verify deletion (GET returns 400):

{
  detail: Response with id resp_ef4879f3-f12d-421a-84ac-d461c68e2767 not found
}

Related Issue

Closes #4336

[github-actions]

✱ Stainless preview builds

This PR will update the llama-stack-client SDKs with the following commit message.

feat: Refactor Agents API to use FastAPI Router

Edit this comment to update it. It will appear in the SDK's changelogs.

✅ llama-stack-client-node studio · code · diff

Your SDK built successfully.
generate ⚠️ → build ✅ → lint ✅ → test ✅

npm install https://pkg.stainless.com/s/llama-stack-client-node/0e4d34ff8c63e43ea6d773f1c5f82c92f0db162c/dist.tar.gz

✅ llama-stack-client-kotlin studio · code · diff

Your SDK built successfully.
generate ⚠️ → lint ✅ → test ⏳

New diagnostics (2 note)

💡 Java/SchemaUnionDiscriminatorMissing: This union schema has more than one object variant, but no [`discriminator`](https://www.stainless.com/docs/reference/openapi-support#discriminator) property, so deserializing the union may be inefficient or ambiguous.

💡 Java/SchemaUnionDiscriminatorMissing: This union schema has more than one object variant, but no [`discriminator`](https://www.stainless.com/docs/reference/openapi-support#discriminator) property, so deserializing the union may be inefficient or ambiguous.

⏳ llama-stack-client-python studio · code · diff

generate ⚠️ → build ⏳ → lint ⏳ → test ⏳

✅ llama-stack-client-go studio · code · diff

Your SDK built successfully.
generate ⚠️ → lint ❗ → test ❗

go get github.com/stainless-sdks/llama-stack-client-go@417fbac8e8bae354153a1905981d4f6a77a2db15

New diagnostics (3 note)

💡 Schema/EnumHasOneMember: This enum schema has just one member, so it could be defined using [`const`](https://json-schema.org/understanding-json-schema/reference/const).

💡 Go/SchemaUnionDiscriminatorMissing: This union schema has more than one object variant, but no [`discriminator`](https://www.stainless.com/docs/reference/openapi-support#discriminator) property, so deserializing the union may be inefficient or ambiguous.

💡 Go/SchemaUnionDiscriminatorMissing: This union schema has more than one object variant, but no [`discriminator`](https://www.stainless.com/docs/reference/openapi-support#discriminator) property, so deserializing the union may be inefficient or ambiguous.

⏳ These are partial results; builds are still running.

---

This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
Last updated: 2025-12-23 22:15:16 UTC

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @skamenan7 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[skamenan7]

@leseb please take a look. thanks.

[leseb]

@skamenan7 please resolve the conflicts

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @skamenan7 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[leseb]

@skamenan7 many failures in tests.

[mergify]

This pull request has merge conflicts that must be resolved before it can be merged. @skamenan7 please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[skamenan7]

I cleaned this PR up into 4 commits so it’s easy to review end-to-end.
Please review and provide feedback for any changes. Glad to fix or enhance. Thanks!

1. feat: migrate Agents API to FastAPI router
   This is the core refactor: move the Agents API (including /v1/responses) off the legacy @webmethod path and onto the router registry pattern (mirroring how Benchmarks works). It introduces a dedicated agents/ package (api.py protocol, models.py request models, fastapi_routes.py router) and wires it into fastapi_router_registry.py.

2. fix: support streaming responses from router endpoints in library client
   With the router migration, the streaming path for /v1/responses can surface as a FastAPI StreamingResponse (SSE) rather than the previous “async iterator” contract. The library client’s streaming logic was written around the old assumption and could end up iterating the wrong object or falling back into JSON serialization. This change makes the client explicitly handle both cases (AsyncIterator vs StreamingResponse.body_iterator) and consistently emit SSE as bytes.

3. fix: disable OTEL SDK in docker integration tests
   CI docker integration runs were flaky due to OTEL exporters attempting to talk to an OTLP collector (and occasionally causing connection errors / shutdown noise). Telemetry integration is already disabled in tests (Replay tests delayed due to telemetry collector #4089), so for docker-mode I set OTEL_SDK_DISABLED=true to prevent exporter traffic during tests and keep the docker jobs deterministic.

4. fix: dedupe responses input union in Stainless spec
   After the router migration, the OpenAPI request schema for POST /v1/responses shows up under CreateResponseRequest.input and ends up including a duplicated OpenAIResponseMessage-Input variant inside nested unions. Stainless codegen is sensitive to this and was failing with Go name clashes and Python duplicate declarations. I didn’t want to change the stable spec (oasdiff would treat that as breaking), so the fix is applied only to the combined/stainless spec generation: dedupe redundant sibling $ref entries during OpenAPI generation and regenerate the Stainless artifacts.

    cc: @leseb