Skip to content

Feature/add phi4 convo#14

Merged
EricApgar merged 13 commits intomainfrom
feature/add-phi4-convo
Mar 15, 2026
Merged

Feature/add phi4 convo#14
EricApgar merged 13 commits intomainfrom
feature/add-phi4-convo

Conversation

@EricApgar
Copy link
Owner

Mainly got the Phi4 model up to date with a similar use as the GPT model.

Closing out some issues that were technically closed by older PR's.

@EricApgar EricApgar self-assigned this Mar 8, 2026
@EricApgar EricApgar added the enhancement New feature or request label Mar 8, 2026
@EricApgar
Copy link
Owner Author

EricApgar commented Mar 15, 2026

Test suite added

Added a pytest test suite and GitHub Actions workflow as part of this PR.

New files:

  • tests/test_gpt_oss_20b.py — loads the real GPT model and calls ask(), asserting a non-empty string response
  • tests/test_phi4_multimodal_instruct.py — loads the real Phi4 model and calls ask() with a synthetic image, asserting a non-empty string response
  • .github/workflows/tests.yml — triggers on PRs and pushes to main

Key design decisions:

  • Tests call the real models rather than mocking, since LLM output consistency can't be tested — only that the model runs without errors
  • Model cache path is passed via LLM_MODEL_CACHE as an inline environment variable (e.g. LLM_MODEL_CACHE=/path/to/cache pytest) so it doesn't persist in the shell
  • Each model fixture uses scope='module' to load the model once per file, then tears down with del + torch.cuda.empty_cache() so both models can be tested sequentially without exceeding GPU memory
  • Tests skip automatically if LLM_MODEL_CACHE is not set, so CI doesn't hard-fail without model weights
  • The workflow is configured for a self-hosted runner (a machine with a GPU and local model weights). See Settings → Actions → Runners to register one, and add LLM_MODEL_CACHE as a repository secret.

To run locally:

uv sync --extra all
LLM_MODEL_CACHE=/home/yourname/Repos/model_cache pytest

@EricApgar EricApgar merged commit 245c09b into main Mar 15, 2026
1 check failed
@EricApgar EricApgar deleted the feature/add-phi4-convo branch March 15, 2026 02:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add conversation structures. Add check to make sure model is loaded before using. Restrict model specific libraries to specific installs

1 participant