[CLI] Foundry Local 0.10.2 only exposes CPU model variants although CUDA Execution Provider is installed

[dahaii]

Problem

Foundry Local correctly detects my NVIDIA RTX 3060 Laptop GPU and successfully downloads and registers all available execution providers during server startup, including:

• CUDAExecutionProvider
• NvTensorRTRTXExecutionProvider
• OpenVINOExecutionProvider
• WebGpuExecutionProvider

The Foundry server starts successfully, models can be downloaded, loaded and executed without any errors.

However, every model exposed by the Azure Foundry catalog is CPU-only.

Examples:

• phi-4-mini → Device: CPU
• phi-3.5-mini → Device: CPU
• qwen3-embedding-0.6b → Device: CPU

Running:

foundry model list -o json

shows that every available model has:

"device": "Cpu"

Additionally, the daemon log reports:

Device: CPU, EPs: CPUExecutionProvider

This seems inconsistent because CUDAExecutionProvider is successfully downloaded and registered during server startup.

Expected behavior

If GPU model variants are supported for my hardware, I would expect Foundry Local to expose them automatically.

Is there an additional configuration required, or does Foundry Local CLI 0.10.2 currently expose only CPU model variants?

Diagnostics

Additional information

• NVIDIA driver is installed correctly.
• nvidia-smi detects the RTX 3060 without any issues.
• foundry status reports no warnings.
• Models execute successfully on CPU.
• The Azure Foundry model catalog contains only CPU variants on my installation.

System

• OS: Microsoft Windows 10.0.26200
• Architecture: X64
• CPU: 11th Gen Intel(R) Core(TM) i7-11800H @ 2.30GHz (16 logical cores)
• GPU: Intel Corporation Intel(R) UHD Graphics, NVIDIA GeForce RTX 3060 Laptop GPU (6 GB)
• RAM: 16 GB
• Disk: 212.9 GB free / 930.3 GB total (C:)
• .NET: .NET 9.0.18

Service

• State: Ready
• CLI version: 0.10.2

Models

• Available variants: 46
• Loaded: 1
• Cached: 1

Connectivity

• Local service: Reachable
• Model registry: Reachable

Warnings

• No warnings.

[FALLEN-01]

Version 10.1 shows gpu version

[natke]

Thank you reporting this @dahaii. We have root caused the issue and will have a new version out ASAP