Suggestion: Add a lab for running inference of model using a KitOps Modelkit instead of Huggingace

[shivaylamba]

Motivation:

The current labs downloads models directly from Hugging Face at runtime. While this is convenient for getting started, many production and enterprise environments require models to be packaged, versioned, and distributed through internal registries rather than fetched from external repositories.

This guide would demonstrate how to replace the runtime Hugging Face download with a KitOps ModelKit. The model is pulled into the Pod via a KitOps initContainer and served locally by vLLM or SGLang, eliminating the need for vllm serve or runtime Hugging Face downloads.

This approach provides several benefits:

• Demonstrates a production-ready model delivery workflow using OCI artifacts.
• Enables reproducible, versioned model deployments.
• Shows how HAMi integrates with modern AI supply chain tooling beyond the default Hugging Face workflow.

[shivaylamba]

This is just a suggestion. Would love to know any feedback from the maintainers, if we can proceed with implementing this.

This could also be a sub section in either vLLM / SGLang lab as a small - markdown toggle note.

[mesutoezdil]

This is just a suggestion. Would love to know any feedback from the maintainers, if we can proceed with implementing this.

This could also be a sub section in either vLLM / SGLang lab as a small - markdown toggle note.

good idea! are yu planning to build this 1 yourself, or is this just a suggestion for sb else to pick up?

if sb take this 1, pls follow: tutorials/_template.md: every command and output should come from a real run, plus a chinese translation, sidebar entry (yeah, we have always an issue), and overview mention

[shivaylamba]

Thanks @mesutoezdil yes planning to build one myself.

[saiyam1814]

Good one @shivaylamba , looking forward to this lab!