Skip to content

Conversation

@r-bit-rry
Copy link
Contributor

Migrates the Eval API from @webmethod decorators to FastAPI routers, following the pattern established by Batches (#4191) and Benchmarks (#4309) migrations.

Changes

  • New module structure: src/llama_stack_api/eval/ with api.py, models.py, fastapi_routes.py
  • Removed: Old src/llama_stack_api/eval.py with @webmethod decorators
  • Updated: Provider implementations to use request models

Migrated Endpoints

Method Route
POST /eval/benchmarks/{benchmark_id}/jobs
POST /eval/benchmarks/{benchmark_id}/evaluations
GET /eval/benchmarks/{benchmark_id}/jobs/{job_id}
DELETE /eval/benchmarks/{benchmark_id}/jobs/{job_id}
GET /eval/benchmarks/{benchmark_id}/jobs/{job_id}/result

Testing

  • All pre-commit hooks pass
  • OpenAPI specs regenerated
  • Type checking verified

Closes #4345

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Dec 22, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Dec 22, 2025

✱ Stainless preview builds

This PR will update the llama-stack-client SDKs with the following commit message.

feat(api): migrate Eval API to FastAPI router (#4345)

Edit this comment to update it. It will appear in the SDK's changelogs.

llama-stack-client-node studio · code · diff

Your SDK built successfully.
generate ⚠️build ✅lint ✅test ✅

npm install https://pkg.stainless.com/s/llama-stack-client-node/4ee47718f8dd8e33d28764f47f5b807592103570/dist.tar.gz
llama-stack-client-kotlin studio

Code was not generated because there was a fatal error.

llama-stack-client-python studio · code · diff

Your SDK built successfully.
generate ⚠️build ✅lint ✅test ✅

pip install https://pkg.stainless.com/s/llama-stack-client-python/1f0211f983f81240a852ca63864849c34c13f3ca/llama_stack_client-0.4.0a14-py3-none-any.whl
llama-stack-client-go studio · code · diff

Your SDK built successfully.
generate ❗lint ❗test ❗

go get github.com/stainless-sdks/llama-stack-client-go@6a76d978255f647b600fc0ceb7f4897ba4717416

⏳ These are partial results; builds are still running.


This comment is auto-generated by GitHub Actions and is automatically kept up to date as you push.
Last updated: 2025-12-23 10:33:24 UTC

@mergify
Copy link

mergify bot commented Dec 22, 2025

This pull request has merge conflicts that must be resolved before it can be merged. @r-bit-rry please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Dec 22, 2025
@r-bit-rry r-bit-rry force-pushed the feature/fastapi-router-migration branch from b343911 to 1870c46 Compare December 22, 2025 19:07
@mergify mergify bot removed the needs-rebase label Dec 22, 2025
},
)
async def run_eval(
benchmark_id_request: Annotated[BenchmarkIdRequest, Depends(get_benchmark_id_request)],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to reviewers, I wanted to use the Annotated for the entire request but the create_path_dependency function only works with single-field models, JobStatusRequest, JobCancelRequest, and JobResultRequest each have two path parameters (benchmark_id and job_id).

@r-bit-rry
Copy link
Contributor Author

@leseb Continuing on our effort here

@mergify
Copy link

mergify bot commented Dec 30, 2025

This pull request has merge conflicts that must be resolved before it can be merged. @r-bit-rry please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Dec 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot. needs-rebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Convert eval API to use a FastAPI router

1 participant