+
+
+
+
+
+ 
+
+ 
+
+ 
+
-`. The LoRA and full fine-tuning scripts use **databricks/databricks-dolly-15k** (general instruction/response pairs), so the exact task varies by script; the idea is the same - adapt the model to your chosen dataset and format. + +Below is a summary of the available training methods. Each method links to its script and provides a brief description for choosing the right approach. + +| Script | Method | Description | Typical VRAM | Recommended For | +|-----------------------------------|-------------------|---------------------------------------------------------------------------------------------------------------------|--------------|-------------------------------------------------| +| [`train_lora.py`](assets/train_lora.py) | **LoRA** | Trains small adapter matrices while freezing base model. 3–5x faster; ~95–98% full quality. | 24–32GB | Advanced users; multiple adapters; more VRAM | +| [`train_qlora.py`](assets/train_qlora.py) *(Linux only)* | **QLoRA** | 4-bit quantization + LoRA adapters. Lowest memory use, fastest, small quality trade-off. Requires `bitsandbytes` (Linux only). | 12–16GB | Most users; fast experiments; limited VRAM | +| [`train_full_finetuning.py`](assets/train_full_finetuning.py) | **Full Fine-tuning** | Updates all model parameters. Maximum quality; highest memory and compute usage. | 40GB+ | Maximum quality; research; large VRAM | + --- ## Understanding the Techniques diff --git a/playbooks/supplemental/pytorch-kernels/README.md b/playbooks/supplemental/pytorch-kernels/README.md index 4c6ecd9b..47ac7053 100644 --- a/playbooks/supplemental/pytorch-kernels/README.md +++ b/playbooks/supplemental/pytorch-kernels/README.md @@ -9,8 +9,6 @@ SPDX-License-Identifier: MIT > This playbook uses special tags that GitHub cannot render. Please visit [amd.com/playbooks](https://amd.com/playbooks) to correctly preview this content. -# Compile your own GPU kernels for PyTorch + AMD ROCm™ Software - ## Overview Write a GPU kernel from scratch, compile it, launch it on an AMD GPU, and watch utilization spike. This playbook shows how GPU computation actually works: write the kernel code, and execute it in parallel across thousands of threads. @@ -103,24 +101,34 @@ AMD GPUs execute threads in groups of **32** called **wavefronts**. All threads --- -## Installing Software Prerequisites +### PyTorch + AMD/HIP -### Create a Virtual Environment +PyTorch ships a ROCm build where the CUDA API surface (`torch.cuda.*`) is transparently backed by HIP. This means: + +- `torch.cuda.is_available()` works on AMD GPUs with ROCm +- `tensor.to("cuda")` allocates on the AMD GPU +- `torch.version.hip` exposes the HIP version + +PyTorch also exposes `torch.cuda._compile_kernel()`, a high-level shortcut to JIT-compile a raw kernel string and get back a callable, without needing a separate build step. + +--- - -On Windows, open a terminal in the directory of your choice and follow the commands to create a venv with ROCm+Pytorch already installed. -```bash -python -m venv kernel-env --system-site-packages -kernel-env\Scripts\activate -``` +## Check for Software Updates -> **Tip**: Windows users may need to modify their PowerShell Execution Policy (e.g. -> setting it to RemoteSigned or Unrestricted) before running some Powershell commands. + + +## Installing Software Prerequisites + +### Prerequisites - Windows +- Install latest: [AMD Adrenalin Software](https://www.amd.com/en/products/software/adrenalin.html) +### Create a Virtual Environment + + On Linux, open a terminal in the directory of your choice and follow the commands to create a venv with ROCm+Pytorch already installed. ```bash sudo apt update @@ -128,10 +136,25 @@ sudo apt install -y python3-venv python3 -m venv kernel-env --system-site-packages source kernel-env/bin/activate ``` - +**Grant your user access to GPU devices** (log out and back in for this to take effect): + +```bash +sudo usermod -aG render,video $LOGNAME +``` + +On Linux, open a terminal in the directory of your choice and follow the commands to create a venv. +```bash +sudo apt update +sudo apt install -y python3-venv +python3 -m venv kernel-env +source kernel-env/bin/activate +``` + + + On Windows, open a terminal in the directory of your choice and follow the commands to create a venv. ```powershell @@ -144,17 +167,6 @@ kernel-env\Scripts\activate - -On Linux, open a terminal in the directory of your choice and follow the commands to create a venv. -```bash -sudo apt update -sudo apt install -y python3-venv -python3 -m venv kernel-env -source kernel-env/bin/activate -``` - - - ### Installing Basic Dependencies diff --git a/playbooks/supplemental/pytorch-kernels/playbook.json b/playbooks/supplemental/pytorch-kernels/playbook.json index 3389fdac..69bfcd28 100644 --- a/playbooks/supplemental/pytorch-kernels/playbook.json +++ b/playbooks/supplemental/pytorch-kernels/playbook.json @@ -2,7 +2,7 @@ "id": "pytorch-kernels", "title": "Custom GPU Kernels with PyTorch and AMD ROCm\u2122", "description": "Write and optimize custom GPU kernels using PyTorch and AMD ROCm\u2122 software on AMD Ryzen\u2122 AI", - "time": 120, + "time": 60, "supported_platforms": { "halo": [ "linux" diff --git a/playbooks/supplemental/speech2speech-translation/README.md b/playbooks/supplemental/speech2speech-translation/README.md index 5fcc05ca..5a5725ae 100644 --- a/playbooks/supplemental/speech2speech-translation/README.md +++ b/playbooks/supplemental/speech2speech-translation/README.md @@ -9,8 +9,6 @@ SPDX-License-Identifier: MIT > This playbook uses special tags that GitHub cannot render. Please visit [amd.com/playbooks](https://amd.com/playbooks) to correctly preview this content. -# Live Speech2Speech Translation on AMD Radeon™ GPU - ## Overview The AMD ROCm™ software and PyTorch stack create a unified ecosystem for on-device AI. It works for both Windows and Linux with official support for a wide range of devices including Ryzen™ AI APUs and Radeon™ GPUs. @@ -29,28 +27,23 @@ This playbook will teach you how to run low-latency, expressive, and private spe - Conveys tone, emotion, and intent without awkward pauses - Enables global collaboration and faster decision-making -## Setting Up Your Environment +## Setting the Memory Configuration -### Create a Virtual Environment + - -On Windows, open a terminal in the directory of your choice and follow the commands to create a venv with ROCm+Pytorch already installed: +## Check for Software Updates +> **Note**: If VSCode is not installed, you can install it with Ryzen AI Developer Center. - -```bash -python -m venv s2st-env --system-site-packages -s2st-env\Scripts\activate -``` - - + + -> **Tip**: Windows users may need to modify their PowerShell Execution Policy (e.g. -> setting it to RemoteSigned or Unrestricted) before running some Powershell commands. +## Installing Software Prerequisites - +### Create a Virtual Environment + On Linux, open a terminal and run the following prompt to create a venv with ROCm+Pytorch already installed: @@ -62,16 +55,36 @@ source s2st-env/bin/activate ``` - +**Grant your user access to GPU devices** (log out and back in for this to take effect): + +```bash +sudo usermod -aG render,video $LOGNAME +``` + +On Linux, open a terminal and run the following prompt to create a venv: + + +```bash +sudo apt update +sudo apt install -y python3-venv +python3 -m venv s2st-env +source s2st-env/bin/activate +``` + + + + + -On Windows, open a terminal in the directory of your choice and follow the commands to create a venv: + +On Windows, open a terminal in the directory of your choice and follow the commands to create a venv with ROCm+Pytorch already installed: ```bash -python -m venv s2st-env +python -m venv s2st-env --system-site-packages s2st-env\Scripts\activate ``` @@ -80,26 +93,28 @@ s2st-env\Scripts\activate > **Tip**: Windows users may need to modify their PowerShell Execution Policy (e.g. > setting it to RemoteSigned or Unrestricted) before running some Powershell commands. - + - -On Linux, open a terminal and run the following prompt to create a venv: + +On Windows, open a terminal in the directory of your choice and follow the commands to create a venv: - + ```bash -sudo apt update -sudo apt install -y python3-venv -python3 -m venv s2st-env -source s2st-env/bin/activate +python -m venv s2st-env +s2st-env\Scripts\activate ``` - - + + +> **Tip**: Windows users may need to modify their PowerShell Execution Policy (e.g. +> setting it to RemoteSigned or Unrestricted) before running some Powershell commands. + + ### Installing Basic Dependencies - + ### Additional Dependencies diff --git a/playbooks/supplemental/speech2speech-translation/playbook.json b/playbooks/supplemental/speech2speech-translation/playbook.json index 08c82cf9..711c95f1 100644 --- a/playbooks/supplemental/speech2speech-translation/playbook.json +++ b/playbooks/supplemental/speech2speech-translation/playbook.json @@ -1,8 +1,8 @@ { "id": "speech2speech-translation", - "title": "Speech-to-Speech Translation", + "title": "Real-Time Speech-to-Speech Translation", "description": "Build a real-time speech-to-speech translation on your local hardware.", - "time": 20, + "time": 30, "supported_platforms": { "halo": [ "linux", diff --git a/playbooks/supplemental/unsloth-llms-finetuning/README.md b/playbooks/supplemental/unsloth-llms-finetuning/README.md index 74b3f281..f1300582 100644 --- a/playbooks/supplemental/unsloth-llms-finetuning/README.md +++ b/playbooks/supplemental/unsloth-llms-finetuning/README.md @@ -31,6 +31,17 @@ In this playbook, we use Unsloth together with **LoRA-based SFT**. That means th Unsloth also supports other training approaches, including QLoRA and reinforcement learning workflows. This playbook focuses on the simplest path first: a small LoRA fine-tuning example that users can run, understand, and extend. +## Setting the Memory Configuration + + + + +## Check for Software Updates +> **Note**: If VSCode is not installed, you can install it with Ryzen AI Developer Center. + + + + ## Installing Software Prerequisites ### Create a Virtual Environment @@ -71,16 +82,26 @@ source unsloth-env/bin/activate > **Note:** Python 3.13 is required for Windows. + +Open a PowerShell terminal and create a virtual environment: + +```powershell +python -m venv unsloth-env --system-site-packages +.\unsloth-env\Scripts\activate +``` + + + Open a PowerShell terminal and create a virtual environment: ```powershell -python -m venv unsloth_env -.\unsloth_env\Scripts\activate +python -m venv unsloth-env +.\unsloth-env\Scripts\activate ``` + ### Installing Basic Dependencies - diff --git a/playbooks/supplemental/unsloth-llms-finetuning/playbook.json b/playbooks/supplemental/unsloth-llms-finetuning/playbook.json index 0196e58b..46a84f05 100644 --- a/playbooks/supplemental/unsloth-llms-finetuning/playbook.json +++ b/playbooks/supplemental/unsloth-llms-finetuning/playbook.json @@ -1,8 +1,8 @@ { "id": "unsloth-llms-finetuning", - "title": "Optimized LLMs Fine-tuning with Unsloth", + "title": "Fine-Tune LLMs with Unsloth", "description": "Use Unsloth for memory-efficient fine-tuned LLMs™", - "time": 60, + "time": 45, "unsupported_platforms": { }, "supported_platforms": { @@ -37,7 +37,7 @@ "windows" ] }, - "difficulty": "advanced", + "difficulty": "intermediate", "isNew": false, "isFeatured": false, "developed": true, diff --git a/playbooks/supplemental/vllm-inference/README.md b/playbooks/supplemental/vllm-inference/README.md index b685da88..cdebd0e6 100644 --- a/playbooks/supplemental/vllm-inference/README.md +++ b/playbooks/supplemental/vllm-inference/README.md @@ -10,21 +10,31 @@ SPDX-License-Identifier: MIT -# High-Performance LLM Inference with vLLM - ## Overview vLLM is a high-performance inference engine designed for large language models (LLMs). It provides optimized serving with continuous batching for high throughput and an OpenAI-compatible API for seamless application integration. This makes vLLM great for production deployments where speed and resource efficiency are critical. This playbook teaches you how to serve LLMs using containerized vLLM on the integrated GPU and interact with models through the OpenAI Python API. -## In This Playbook, You Will Learn +## What You'll Learn - How to set up and start a vLLM server with ROCm support - How to interact with models via OpenAI-compatible API endpoints - How to send prompts to the local server with `vllm-prompt` -## Starting vLLM +## Setting the Memory Configuration + + + + +## Check for Software Updates + +> **Note**: If VSCode is not installed, you can install it with Ryzen AI Developer Center. + + + + +## Installing Software Prerequisites This playbook uses a prebuilt container image that includes vLLM, ROCm support, and the helper scripts needed to launch the server. You do not need to install PyTorch, vLLM, or local playbook scripts manually. diff --git a/playbooks/supplemental/vllm-inference/playbook.json b/playbooks/supplemental/vllm-inference/playbook.json index 373fe4e4..e92cc9e8 100644 --- a/playbooks/supplemental/vllm-inference/playbook.json +++ b/playbooks/supplemental/vllm-inference/playbook.json @@ -1,8 +1,8 @@ { "id": "vllm-inference", - "title": "Quick Start on vLLM", + "title": "Getting Started with vLLM", "description": "Learn how to run inference and serving using containerized vLLM on the integrated GPU", - "time": 90, + "time": 30, "unsupported_platforms": { "halo": [ { "platform": "windows", "reason": "vLLM is not supported on Windows" } diff --git a/website/src/app/playbooks/[id]/page.tsx b/website/src/app/playbooks/[id]/page.tsx index bab83c37..d9f3a1f6 100644 --- a/website/src/app/playbooks/[id]/page.tsx +++ b/website/src/app/playbooks/[id]/page.tsx @@ -213,7 +213,7 @@ function HaloPreinstalledDropdown({ - Already pre-installed on your AMD Halo Developer Platform! + Already pre-installed on your AMD Ryzen AI Developer Platform!