serving-llms-on-epyc: behavioral tests and supporting checks - #123
serving-llms-on-epyc: behavioral tests and supporting checks#123amd-lalithnc wants to merge 2 commits into
Conversation
…chat-template endpoint selection Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com> Change-Id: I6cf00350b723b9254efb1aa506b07d310d3f8ea2
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com> Change-Id: I263b98e578fc183fa7d3cd00052b220d44d5ffce
shailensobhee
left a comment
There was a problem hiding this comment.
Approving. I re-ran the gates locally rather than relying on the PR body; details below.
Verified locally (commit c9bb6da)
-
Structural gate:
.github/scripts/check.sh->0 error(s) across 6 skill(s), plus "Cursor marketplace manifest is up to date" and "Codex plugin and marketplace manifests are up to date". No derived manifest was hand-edited. -
Staleness check: the branch is 7 commits behind
main(includinglemonade-router-builderfrom #110), so I applied this PR's content onto currentmainand ran main's validator against it:0 error(s) across 7 skill(s). The branch being behind is not breaking anything. -
Unit tests: 19/19 of the pure (non-LLM) tests pass. Detection, the Venice stack-compatibility verdicts, template classification, and endpoint selection all behave as documented.
-
Scripts run live:
detect.pyandvalidate.pyboth emit correct JSON.check_model.pywas run against the real HF + vLLM registry:Qwen/Qwen3-0.6B->chat_template.status: present,primary_endpoint: chat_completionsallenai/OLMo-2-0425-1B->chat_template.status: absent,primary_endpoint: completions
So the registry parsing and the base-vs-chat split work against the real source, not just in the unit tests.
-
New image pin exists:
amdih/zendnn_zentorch:vllm_v0.25.1_zentorch_v2.11.0.3_ubuntu22.04_2026_ww30resolves on Docker Hub (digestsha256:027bf7bcdf021ff52211c98514318c7b2dfc95ba9342943367c46c7ca90eb156, pushed 2026-07-29).vllm_versionand the image tag were updated together, andtest_default_stack_pin_is_consistentenforces that going forward. -
Docs to code: no mismatch found.
primary_endpointandsupported_endpointsare genuinely emitted (check_model.py:235), matching whatSKILL.mdandreference.mdtell the agent to read.
On the red checks
The behavioral jobs are failing for CI-infrastructure reasons, not because of this PR. Every one of the 25 errors is the same message:
Failed: claude API not reachable -- are you on the right network?
(... "result":"Not logged in - Please run /login" ...)
That comes from the session-scoped autouse preflight in eval/behavioral/conftest.py, a shared file this PR does not touch. Because the fixture is session-scoped, it errors out the entire module, including the 20 tests that need no LLM at all. The same failure also hits lemonade-router-builder, local-ai-use, and local-ai-app-integration, none of which this PR touches. The runner's claude CLI is simply not logged in.
Worth considering as a separate follow-up: scoping that preflight to only the tests that actually call claude (a fixture on the behavioral tests rather than session-autouse) would let the pure unit tests report honestly when the judge is unavailable.
Non-blocking nit
Two em dashes were added in SKILL.md (lines 352 and 357, the /dev/shm bullet). Not blocking, and consistent with existing prose elsewhere in the repo. Flagging only for house-style consistency.
Nice catch on the --shm-size / --ipc=host conflict, and it is applied consistently across data/epyc.json, SKILL.md, and reference.md.
Split out of #83 so the walkthrough can be reviewed separately. Adds the serving-llms-on-epyc behavioral and unit tests and the code they cover: EPYC 9000-series (Genoa/Turin/Venice) detection, the vLLM 0.25.1 image pin with a runtime stack-compatibility check, and chat-template-based endpoint selection.