Skip to content

serving-llms-on-epyc: behavioral tests and supporting checks - #123

Open
amd-lalithnc wants to merge 2 commits into
amd:mainfrom
amd-lalithnc:serving-llms-on-epyc-behavioral-tests
Open

serving-llms-on-epyc: behavioral tests and supporting checks#123
amd-lalithnc wants to merge 2 commits into
amd:mainfrom
amd-lalithnc:serving-llms-on-epyc-behavioral-tests

Conversation

@amd-lalithnc

Copy link
Copy Markdown
Contributor

Split out of #83 so the walkthrough can be reviewed separately. Adds the serving-llms-on-epyc behavioral and unit tests and the code they cover: EPYC 9000-series (Genoa/Turin/Venice) detection, the vLLM 0.25.1 image pin with a runtime stack-compatibility check, and chat-template-based endpoint selection.

…chat-template endpoint selection

Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Change-Id: I6cf00350b723b9254efb1aa506b07d310d3f8ea2
Signed-off-by: Lalithnarayan C <Lalithnarayan.C@amd.com>
Change-Id: I263b98e578fc183fa7d3cd00052b220d44d5ffce
@Mahdi-CV Mahdi-CV assigned archakamk and shailensobhee and unassigned archakamk Aug 6, 2026

@shailensobhee shailensobhee left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I re-ran the gates locally rather than relying on the PR body; details below.

Verified locally (commit c9bb6da)

  • Structural gate: .github/scripts/check.sh -> 0 error(s) across 6 skill(s), plus "Cursor marketplace manifest is up to date" and "Codex plugin and marketplace manifests are up to date". No derived manifest was hand-edited.

  • Staleness check: the branch is 7 commits behind main (including lemonade-router-builder from #110), so I applied this PR's content onto current main and ran main's validator against it: 0 error(s) across 7 skill(s). The branch being behind is not breaking anything.

  • Unit tests: 19/19 of the pure (non-LLM) tests pass. Detection, the Venice stack-compatibility verdicts, template classification, and endpoint selection all behave as documented.

  • Scripts run live: detect.py and validate.py both emit correct JSON. check_model.py was run against the real HF + vLLM registry:

    • Qwen/Qwen3-0.6B -> chat_template.status: present, primary_endpoint: chat_completions
    • allenai/OLMo-2-0425-1B -> chat_template.status: absent, primary_endpoint: completions

    So the registry parsing and the base-vs-chat split work against the real source, not just in the unit tests.

  • New image pin exists: amdih/zendnn_zentorch:vllm_v0.25.1_zentorch_v2.11.0.3_ubuntu22.04_2026_ww30 resolves on Docker Hub (digest sha256:027bf7bcdf021ff52211c98514318c7b2dfc95ba9342943367c46c7ca90eb156, pushed 2026-07-29). vllm_version and the image tag were updated together, and test_default_stack_pin_is_consistent enforces that going forward.

  • Docs to code: no mismatch found. primary_endpoint and supported_endpoints are genuinely emitted (check_model.py:235), matching what SKILL.md and reference.md tell the agent to read.

On the red checks

The behavioral jobs are failing for CI-infrastructure reasons, not because of this PR. Every one of the 25 errors is the same message:

Failed: claude API not reachable -- are you on the right network?
(... "result":"Not logged in - Please run /login" ...)

That comes from the session-scoped autouse preflight in eval/behavioral/conftest.py, a shared file this PR does not touch. Because the fixture is session-scoped, it errors out the entire module, including the 20 tests that need no LLM at all. The same failure also hits lemonade-router-builder, local-ai-use, and local-ai-app-integration, none of which this PR touches. The runner's claude CLI is simply not logged in.

Worth considering as a separate follow-up: scoping that preflight to only the tests that actually call claude (a fixture on the behavioral tests rather than session-autouse) would let the pure unit tests report honestly when the judge is unavailable.

Non-blocking nit

Two em dashes were added in SKILL.md (lines 352 and 357, the /dev/shm bullet). Not blocking, and consistent with existing prose elsewhere in the repo. Flagging only for house-style consistency.

Nice catch on the --shm-size / --ipc=host conflict, and it is applied consistently across data/epyc.json, SKILL.md, and reference.md.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants