Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 67 additions & 0 deletions agent-skills/quark-install/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
---
name: quark-install
description: Install, verify, or repair AMD Quark environments for PyTorch or ONNX workflows. Use for amd-quark setup, Python and accelerator compatibility checks, ROCm/CUDA/CPU package selection, Quark import failures, ONNX Runtime provider selection, or first-run kernel and custom-operator compilation. Inspect first and require confirmation before changing packages or system dependencies.
---

# Install AMD Quark

Prepare a reproducible Quark environment without guessing the user's accelerator, Python environment, or package source. This skill is self-contained: do not delegate its steps to another skill.

## Workflow

1. Clarify the intended flow: PyTorch, ONNX-to-ONNX, or both.
2. Collect facts before proposing commands:

```bash
python scripts/collect_environment.py --output env_context.json
```

Also ask which Python environment may be modified and whether CPU fallback is acceptable.
3. Read [references/install-options.md](references/install-options.md). Recheck the release-matched [official installation guide](https://quark.docs.amd.com/latest/install.html) when generating commands; package and accelerator matrices change.
4. Present one installation plan containing:
- environment name and Python executable;
- PyTorch or ONNX Runtime variant and index;
- Quark wheel source;
- compiler or first-import requirements;
- exact install and verification commands.
5. Get explicit confirmation before installing, uninstalling, upgrading, compiling, or modifying system packages.
6. Execute only the approved commands. Capture command output and package versions.
7. Run the relevant verification checks from [references/verification-and-recovery.md](references/verification-and-recovery.md).
8. Write `quark_install_result.json` with status, versions, backend, commands run, and per-check results.

## Decision rules

- Prefer the universal PyPI wheel when compatibility is uncertain. Use a pre-built AMD-index wheel only after confirming its Python, PyTorch, OS, and accelerator match.
- Install a GPU-enabled PyTorch build from the backend-specific PyTorch index. Do not use bare `pip install torch` for an intended GPU environment.
- Treat absent GPU tools as `unknown`, not automatically as CPU-only. Ask before choosing CPU.
- Never mix a CUDA PyTorch build with a ROCm environment, or the reverse.
- Install exactly one ONNX Runtime variant. On current ROCm 7.x guidance, ONNX Runtime may use the CPU package; explain that trade-off instead of inventing an unsupported ROCm wheel.
- Do not claim success from `pip` alone. Imports and backend/provider checks must pass.
- Do not silently repair an existing environment. Show the proposed uninstall/reinstall sequence and its impact first.

## Result contract

Record enough evidence to reproduce or diagnose the environment:

```json
{
"status": "ok",
"python": "3.13.0",
"environment": "/path/to/python",
"accelerator": "amd-rocm",
"packages": {"amd-quark": "0.12.0", "torch": "..."},
"verification": {
"quark_import": "pass",
"torch_backend": "pass",
"torch_kernel": "not-run",
"onnx_custom_ops": "not-requested"
},
"commands": []
}
```

Use `failed` when a required check fails and `partial` when optional compilation or a requested backend check remains unverified. Include the exact failing command and error excerpt.

## Stop conditions

Stop before mutation when the active environment is ambiguous, the requested backend conflicts with detected packages, required credentials are missing, or a wheel compatibility cannot be established. Return the evidence collected and the smallest question or manual check needed to proceed.
3 changes: 3 additions & 0 deletions agent-skills/quark-install/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
interface:
display_name: "Install AMD Quark"
short_description: "Plan, install, and verify a Quark environment"
53 changes: 53 additions & 0 deletions agent-skills/quark-install/references/install-options.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Installation options

Use this as a planning checklist, then verify current commands against the official installation guide for the Quark release the user will run.

## Environment prerequisites

- Quark 0.12 documentation supports Python 3.11, 3.12, and 3.13; Python 3.14 is not supported.
- PyTorch 2.2 or later is required by the release documentation. Accelerator-specific combinations are narrower, so consult the release's `tools/ci/install_torch.sh` when working from a Quark checkout.
- The universal Quark wheel compiles optional fast kernels or ONNX custom operators on first import. Linux needs a C++ compiler such as `g++`; GPU compilation also needs `hipcc` or `nvcc` and its toolkit path.
- Pre-built Quark wheels require a compatible Python, PyTorch, operating system, and accelerator combination.

## Quark package source

| Choice | When to use | Command pattern |
|---|---|---|
| Universal PyPI wheel | Default and widest compatibility | `pip install amd-quark` |
| Pre-built CPU | Matching supported Python and PyTorch; avoid first-run compile | `pip install amd-quark --extra-index-url https://pypi.amd.com/quark/cpu/simple` |
| Pre-built CUDA 12.8 | Matching CUDA/PyTorch environment | `pip install amd-quark --extra-index-url https://pypi.amd.com/quark/cu128/simple` |
| Pre-built ROCm 7.1 | Linux and matching ROCm/PyTorch environment | `pip install amd-quark --extra-index-url https://pypi.amd.com/quark/rocm71/simple` |
| Pre-built ROCm 7.2 | Linux and matching ROCm/PyTorch environment | `pip install amd-quark --extra-index-url https://pypi.amd.com/quark/rocm72/simple` |

These are release-0.12-era options, not a permanent compatibility promise. If a requested combination is absent from the current official selector, use the universal wheel or stop and explain the gap.

## PyTorch backend

Use the [PyTorch installation selector](https://pytorch.org/get-started/locally/) or Quark's release-matched CI matrix. The command pattern is:

```bash
# ROCm: use the confirmed rocmX.Y tag
pip install torch torchvision --index-url https://download.pytorch.org/whl/rocmX.Y

# CUDA: use the confirmed cuXYZ tag
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cuXYZ

# Explicit CPU environment
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
```

Never substitute a remembered tag for a checked compatibility matrix.

## ONNX Runtime

Quark 0.12 documentation requires ONNX Runtime `>=1.22.2,<=1.25.1`.

```bash
# CPU, including the documented ROCm 7.x fallback
pip install "onnxruntime>=1.22.2,<=1.25.1"

# CUDA when the current compatibility guidance supports it
pip install "onnxruntime-gpu>=1.22.2,<=1.25.1"
```

Before changing variants, inspect `python -m pip list` and require approval for removing conflicting `onnxruntime*` packages.
43 changes: 43 additions & 0 deletions agent-skills/quark-install/references/verification-and-recovery.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Verification and recovery

## Required checks

Run checks with the exact Python executable used for installation.

```bash
python -c "import quark; print(quark.__version__)"
python -c "import torch; print(torch.__version__, torch.version.cuda, torch.version.hip); print(torch.cuda.is_available(), torch.cuda.device_count())"
```

For a PyTorch flow, optionally force first-run kernel compilation when requested:

```bash
python -c "import quark.torch.kernel; print('Quark Torch kernels OK')"
```

For an ONNX flow:

```bash
python -c "import onnxruntime as ort; print(ort.__version__, ort.get_available_providers())"
python -c "import quark.onnx; print('quark.onnx OK')"
python -c "import quark.onnx.operators.custom_ops; print('Quark ONNX custom ops OK')"
```

The custom-op check may compile code on first import. Treat it as a mutation/cost-bearing step and confirm first.

## Common recovery paths

| Evidence | Likely cause | Next action |
|---|---|---|
| `No module named quark` | Wrong environment or install failed | Compare `which python` with `python -m pip --version`; reinstall only after confirmation |
| `torch.cuda.is_available()` is false on a requested GPU | CPU wheel or backend mismatch | Inspect `torch.version.cuda` and `torch.version.hip`; select a release-matched backend wheel |
| Torch kernel import cannot compile | Compiler/toolkit unavailable | Check `g++`, `hipcc`/`nvcc`, `ROCM_PATH`/`CUDA_HOME`; consider a compatible pre-built wheel |
| ONNX provider missing | Wrong ORT variant or unsupported backend | Inspect installed `onnxruntime*` distributions and current provider support |
| ONNX custom-op symbol/ABI error | ORT changed after compilation | Confirm compatible ORT pin, clear only the identified build cache, and rebuild after approval |
| Python is outside the supported range | Unsupported dependency set | Create a fresh Python 3.11-3.13 environment; do not force incompatible pins into the old one |

When recovery would replace packages, preserve a snapshot first:

```bash
python -m pip freeze > quark-environment-before.txt
```
87 changes: 87 additions & 0 deletions agent-skills/quark-install/scripts/collect_environment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
#!/usr/bin/env python3
# Copyright (C) 2026, Advanced Micro Devices, Inc. All rights reserved.
# SPDX-License-Identifier: MIT

"""Collect non-mutating facts needed to plan an AMD Quark installation."""

from __future__ import annotations

import argparse
import importlib.metadata
import json
import os
import platform
import shutil
import subprocess
import sys
from pathlib import Path


def distribution_version(*names: str) -> str | None:
for name in names:
try:
return importlib.metadata.version(name)
except importlib.metadata.PackageNotFoundError:
continue
return None


def command_output(command: list[str]) -> str | None:
if not shutil.which(command[0]):
return None
try:
result = subprocess.run(command, capture_output=True, text=True, timeout=10, check=False)
except (OSError, subprocess.TimeoutExpired):
return None
text = (result.stdout or result.stderr).strip()
return text[:2000] if text else None


def collect() -> dict[str, object]:
packages = {
"amd-quark": distribution_version("amd-quark"),
"torch": distribution_version("torch"),
"onnx": distribution_version("onnx"),
"onnxruntime": distribution_version("onnxruntime"),
"onnxruntime-gpu": distribution_version("onnxruntime-gpu"),
"onnxruntime-rocm": distribution_version("onnxruntime-rocm", "onnxruntime_rocm"),
}
tools = {name: shutil.which(name) for name in ("g++", "hipcc", "nvcc", "rocm-smi", "nvidia-smi")}
return {
"python": {
"version": platform.python_version(),
"executable": sys.executable,
"implementation": platform.python_implementation(),
"virtual_env": os.environ.get("VIRTUAL_ENV"),
"conda_prefix": os.environ.get("CONDA_PREFIX"),
},
"platform": {"system": platform.system(), "release": platform.release(), "machine": platform.machine()},
"environment": {
key: os.environ.get(key)
for key in ("ROCM_PATH", "HIP_VISIBLE_DEVICES", "CUDA_HOME", "CUDA_VISIBLE_DEVICES")
},
"packages": packages,
"tools": tools,
"accelerator_evidence": {
"rocm_smi": command_output(["rocm-smi", "--showproductname"]),
"nvidia_smi": command_output(
["nvidia-smi", "--query-gpu=name,memory.total,driver_version", "--format=csv,noheader"]
),
},
}


def main() -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--output", type=Path, help="Write JSON to this path instead of stdout")
args = parser.parse_args()
payload = json.dumps(collect(), indent=2, sort_keys=True) + "\n"
if args.output:
args.output.write_text(payload, encoding="utf-8")
else:
print(payload, end="")
return 0


if __name__ == "__main__":
raise SystemExit(main())
82 changes: 82 additions & 0 deletions agent-skills/quark-onnx-ptq/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
---
name: quark-onnx-ptq
description: Plan, run, validate, or debug AMD Quark ONNX-to-ONNX post-training quantization. Use for static or dynamic ONNX quantization, INT8/INT4/BF16/BFP/MX formats, calibration data readers, QConfig and ModelQuantizer, Ryzen AI targets, execution-provider issues, AutoSearch, custom operators, or invalid quantized ONNX artifacts. Covers prerequisites through validation without relying on other skills.
---

# Quantize ONNX Models with AMD Quark

Build a reproducible ONNX PTQ workflow around a pinned model, representative data, target execution provider, and measurable acceptance criteria.

## Workflow

### 1. Intake and inspect

Collect:

- input `.onnx` path, external-data files, model size, opset, and expected input shapes/dtypes;
- deployment target (CPU, CUDA, ROCm, Ryzen AI/NPU, or another runtime) and available execution providers;
- desired format/preset or accuracy, size, and latency goal;
- calibration/evaluation datasets, preprocessing, sample counts, and metric;
- memory, disk, and time budget; output path and overwrite policy.

Inspect the model before planning:

```bash
python scripts/inspect_onnx_artifact.py --quantized-model /path/to/input.onnx
python -c "import onnxruntime as ort; print(ort.__version__, ort.get_available_providers())"
```

The first command labels an unquantized model as a warning, which is expected during intake.

### 2. Verify prerequisites

Use one compatible environment containing Quark, ONNX, and exactly one ONNX Runtime variant. Read [references/workflow-and-presets.md](references/workflow-and-presets.md) and the release-matched [official ONNX guide](https://quark.docs.amd.com/latest/onnx/basic_usage_onnx.html). If custom operators may be needed, explain first-run compilation and confirm before triggering it.

### 3. Build the plan

Specify:

- pinned Quark release and model checksum/path;
- QConfig preset or explicit config, with target-runtime evidence;
- calibration reader behavior, input names/shapes/dtypes, preprocessing, and sample count;
- execution provider and whether CPU fallback is acceptable;
- algorithms, external-data handling, output path, and evaluation threshold;
- estimated peak memory, disk, trial count, and wall time.

Use basic PTQ first unless there is evidence it misses the accuracy target. AutoSearch is an explicit, budgeted escalation; read [references/autosearch-and-validation.md](references/autosearch-and-validation.md).

### 4. Confirm and execute

Show the generated script or exact command. Get explicit confirmation before package changes, downloads, custom-op compilation, quantization, AutoSearch, or overwriting artifacts. Capture stdout/stderr, environment versions, provider list, configuration, and data provenance.

### 5. Validate

1. Run structural and ONNX checker validation:

```bash
python scripts/inspect_onnx_artifact.py \
--source-model /path/to/float.onnx \
--quantized-model /path/to/quantized.onnx \
--output validation_artifacts.json
```

2. Load the quantized artifact with the intended execution provider and confirm the provider actually used.
3. Run deterministic smoke inference.
4. Compare float and quantized outputs/accuracy using identical preprocessing and data.
5. If performance matters, measure size, peak memory, and latency on the target runtime.

Report ONNX checker, quantization-marker, metadata/I-O, provider-load, inference, quality, and performance results separately.

## Rules

- Do not treat provider availability as provider usage; capture both.
- A calibration reader must emit the model's exact input names, shapes, and dtypes and must yield at least one batch.
- Keep calibration and evaluation data distinct in accuracy claims.
- Preserve every external-data file with the model. Never move only the `.onnx` protobuf.
- Do not infer deployment compatibility from an ONNX checker pass or Q/DQ node count.
- Do not start AutoSearch without a trial/time/disk budget and explicit approval.
- On failure, preserve the first causal error. Later cancellation or provider-fallback messages may be secondary.

## Deliverable

Return the pinned inputs, generated config/script, exact command, artifact paths, logs, validation matrix, metric delta, provider evidence, limitations, and smallest recovery step.
3 changes: 3 additions & 0 deletions agent-skills/quark-onnx-ptq/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
interface:
display_name: "Quark ONNX PTQ"
short_description: "Quantize and validate ONNX models with Quark"
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# AutoSearch and validation

## When to use AutoSearch

Use manual/default PTQ first. Escalate only when the measured quality misses an agreed target and multiple configuration choices are plausible.

Before launching a search, define:

- preset or explicit search space;
- calibration reader and optional custom evaluator;
- metric and optimization direction;
- trial count, parallel jobs/devices, wall-time limit, disk estimate, and stop condition;
- persistent output, database, and log paths so a run can be resumed or audited.

Show the full search script and cost estimate, then get explicit confirmation. The [official AutoSearch Pro guide](https://quark.docs.amd.com/latest/onnx/user_guide_auto_search_pro.html) is the authority for the pinned release's API and presets.

Do not compare AutoSearch candidates with different data or preprocessing. If using a built-in L1/L2 proxy, say that it is a proxy rather than task accuracy.

## Validation ladder

1. **Structure:** files exist; external data resolves; protobuf loads.
2. **Schema:** `onnx.checker.check_model` passes.
3. **Quantization evidence:** expected Q/DQ, quantized operators, initializer types, or Quark domains exist.
4. **Metadata/I-O:** expected model inputs and outputs remain compatible.
5. **Runtime load:** the intended ONNX Runtime/provider loads the model without silent fallback.
6. **Smoke inference:** fixed input produces finite outputs of expected shapes.
7. **Quality:** task metric or numerical comparison meets the approved threshold.
8. **Performance:** target-hardware size, memory, and latency meet the goal.

The bundled inspector covers only steps 1-4. State that boundary in every result.

## Recovery clues

| Symptom | Check first |
|---|---|
| Provider missing | Installed `onnxruntime*` variant and `ort.get_available_providers()` |
| Silent CPU fallback | Providers passed to the actual session, not only available providers |
| No calibration data | Reader input names, first batch, rewind, and sample count |
| Model exceeds 2 GB | External-data export and colocated data files |
| Custom op not registered | Quark custom-op build, ABI-compatible ORT, and session registration |
| Accuracy collapse | Float baseline, preprocessing parity, calibration representativeness, excluded nodes, preset/algorithm |
| OOM or disk exhaustion | Batch/sample count, disk cache, worker count, external data, AutoSearch trials |
Loading