onnxruntime-genai does not support VitisAI (NPU) or RyzenAI in public builds despite RyzenAI 1.7.0 installation for RyzenAI RAG

[bmhydar]

Following AMD’s documented RyzenAI RAG workflow and using RyzenAI 1.7.0, the provided LLM examples do not run as provided.
https://github.com/amd/RyzenAI-SW/tree/main/LLM-examples/RAG-OGA
When running the RAG example exactly as documented (updating only the dataset path and model path as instructed), LLM initialization fails. The AMD-distributed LLM model configuration references the provider name RyzenAI, which results in the following error from onnxruntime-genai:

RuntimeError: Unknown provider name 'RyzenAI'

After correcting the provider name in the model configuration to VitisAI (as required by onnxruntime-genai), initialization fails again with:

RuntimeError: VitisAI execution provider is not supported in this build

This indicates that public onnxruntime-genai packages do not include NPU (VitisAI) support, even though:

RyzenAI installs NPU drivers successfully

AMD’s RAG examples and downloadable model configs imply that NPU-backed LLM inference should work

As a result, AMD’s downloadable files and documented directions do not work as-is. Users must either modify AMD-provided configuration files and then fall back to CPU execution, or obtain non-public builds to proceed.

[MathewCNichols]

I'm hitting the same roadblock.

[dwithchenna]

@bmhydar can you share the list of python package installed in your conda environment using conda list. The default RyzenAI conda package should include all the necessary packages.

It is helpful to create a clone of the original conda environment, to avoid corrupting the python package dependencies when install any custom python libraries.

[abhishek-amd]

Hi @bmhydar
Please share the installed package list in your conda environment.

For a fresh run , you can follow this step.
1. create a new conda environment : https://ryzenai.docs.amd.com/en/latest/inst.html
2. create a clone of the original conda environment.
3. activate the cloned environment.
4. cd RyzenAI-SW/example/llm/RAG-OGA : clone the repo and navigate to RAG-OGA folder.
5. git clone https://huggingface.co/amd/Llama-3.2-3B-Instruct-onnx-ryzenai-1.7-hybrid ./model : will clone the hybrid model into model folder of current directory. ( You do not need to update the model path in rag.py).
6. pip install -r requirements.txt
7. python custom_embedding/export_bge_onnx.py : The compiled BGE (BAAI General Embedding) ONNX model will be stored in the cache folder named modelcachekey_bge.
8. python rag.py --direct_llm ( currently you are getting an error when running this command).

If you are getting an error, please share the steps to reproduce it.