v0.7.0

[nvluxiaoz]

TensorRT Edge-LLM 0.7.0 Release 2026-04-28

We are very excited to announce release 0.7.0 of TensorRT Edge-LLM!

• We welcome community contribution and have merged Add NVTX purple color #68 and chore: Implement FindTensorRT module with component support #42. Thanks the contributors @matiaslin and @victoroliv2.
• We are excited to launch our pages on Jetson AI Lab. Please check more tutorials and model deployment guidelines there.

Deprecation Notice

• The experimental workflow is expected to reach full parity of the original workflow. The original tensorrt_edgellm is deprecated and will be removed in 0.8.0.

Key Features

• Introduced Day 0 Support of NVIDIA Nemotron 3 Nano Omni. Please review Jetson AI Lab on how to run TensorRT Edge-LLM for this model.
• Introduced an experimental High Level Python API and OpenAI Compatible Server. Please follow our Quick Start Guide, Introduction and Code.
• Implemented an agent-friendly experimental new workflow to quantize and convert models to ONNX. The quantized checkpoints are now compatible with major frameworks like TensorRT-LLM, vLLM and sgLang, and the ONNX conversion has 0 GPU consumption saves > 70% total memory consumption.
• Expanded pre-quantized models into our model support lists. Please see our updated [supported model lists].(https://nvidia.github.io/TensorRT-Edge-LLM/0.7.0/user_guide/getting_started/supported-models.html)
• Added AGENTS.md to assist development and users. Automatic agents to debug accuracy is added.
• Added performance dashboard for selected models. The dashboard will keep updating for newer versions and models.

Other Important Features

Model Support

• Added Qwen3.5 LLM/VLM support
• Added NVIDIA Nemotron Nano 3 30B A3B NVFP4 support
• Added Qwen3-ASR quantization workflow

Performance Improvements

• Added FP8 embedding support to reduce embedding-table memory
• Reduced runtime memory by sharing TensorRT execution context memory
• Added multi-batch cutedsl prefill kernels
• Improved performance by cutedsl-based Mamba SSD kernels

Runtime Extensions

• Unified LLM runtime execution paths for vanilla and EAGLE3 decoding
• Added per-slot streaming with the StreamChannel API
• Replaced LinearKVCache with per layer HybridCacheManager to support more hybrid models

Workflow Improvements

• Improved build workflow with automatic TensorRT detection and cutedsl kernel binary shipment
• Upgraded Transformers support to 5.x

NVIDIA Contributors

@fans-nv @nvluxiaoz @nvamberl @ruocheng-nv @poweiw @yuanyao-nv @Jasper-NV @charllll @mahu888 @xinrzhang-star @JCalafato @nvmbreughe

---

This discussion was created from the release v0.7.0.