Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/atom-test-trigger.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: ATOM Test

on:
push:
branches: [main]
pull_request:
branches: [main]
types: [opened, synchronize, reopened, ready_for_review]
paths-ignore:
- '**/*.md'
- 'docs/**'
- 'LICENSE'
- '.gitignore'
schedule:
# Nightly at 00:00 Beijing time (16:00 UTC)
- cron: '0 16 * * *'
workflow_dispatch:
inputs:
aiter_branch:
description: 'ROCm/aiter branch to build inside the CI image'
required: false
default: 'main'
type: string

concurrency:
# Keep scheduled main runs from blocking push-triggered validation.
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}

jobs:
atom-test:
uses: ./.github/workflows/atom-test.yaml
with:
checkout_repository: ${{ github.repository }}
checkout_ref: ${{ github.sha }}
source_repo_url: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.clone_url || format('https://github.com/{0}.git', github.repository) }}
source_commit_sha: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.event.head_commit.id || github.sha }}
signal_commit_sha: ${{ github.sha }}
aiter_branch: ${{ github.event_name == 'workflow_dispatch' && github.event.inputs.aiter_branch || 'main' }}
model_scope: ${{ github.event_name == 'schedule' && 'nightly' || github.event_name == 'workflow_dispatch' && 'nightly' || github.event_name == 'push' && 'main' || 'pr' }}
run_precheck: ${{ github.event_name != 'workflow_dispatch' }}
is_draft_pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.draft || false }}
is_fork_pr: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.repo.fork || false }}
publish_dashboard: ${{ github.ref == 'refs/heads/main' && (github.event_name == 'push' || github.event_name == 'schedule') }}
secrets:
github_token: ${{ secrets.GITHUB_TOKEN }}
docker_username: ${{ secrets.DOCKER_USERNAME }}
docker_password: ${{ secrets.DOCKER_PASSWORD }}
amd_hf_token: ${{ secrets.AMD_HF_TOKEN }}
157 changes: 111 additions & 46 deletions .github/workflows/atom-test.yaml
Original file line number Diff line number Diff line change
@@ -1,61 +1,104 @@
name: ATOM Test
name: ATOM Test Reusable

on:
push:
branches: [main]
pull_request:
branches: [main] # Triggers on PRs targeting `main`
types: [opened, synchronize, reopened, ready_for_review]
paths-ignore:
- '**/*.md'
- 'docs/**'
- 'LICENSE'
- '.gitignore'
schedule:
# Nightly at 00:00 Beijing time (16:00 UTC)
- cron: '0 16 * * *'
workflow_dispatch:
workflow_call:
inputs:
checkout_repository:
description: 'Repository to check out into the workflow workspace'
required: false
default: 'ROCm/ATOM'
type: string
checkout_ref:
description: 'Git ref or commit SHA to check out into the workflow workspace'
required: false
default: 'main'
type: string
source_repo_url:
description: 'Clone URL used inside container-side source build flows'
required: false
default: ''
type: string
source_commit_sha:
description: 'Commit SHA used inside container-side source build flows'
required: false
default: ''
type: string
signal_commit_sha:
description: 'Commit SHA used to look up the pre-checkin signal artifact'
required: false
default: ''
type: string
aiter_branch:
description: 'ROCm/aiter branch to build inside the CI image'
required: false
default: 'main'
type: string

concurrency:
# Keep scheduled main runs from blocking push-triggered validation.
group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
model_scope:
description: 'Model scope to run: pr, main, or nightly'
required: false
default: 'pr'
type: string
run_precheck:
description: 'Whether to gate the run on the pre-checkin signal artifact'
required: false
default: false
type: boolean
is_draft_pr:
description: 'Whether the source change is a draft pull request'
required: false
default: false
type: boolean
is_fork_pr:
description: 'Whether the source change comes from a fork'
required: false
default: false
type: boolean
publish_dashboard:
description: 'Whether to publish accuracy results to the benchmark dashboard'
required: false
default: false
type: boolean
secrets:
github_token:
required: false
docker_username:
required: false
docker_password:
required: false
amd_hf_token:
required: false

env:
ATOM_BASE_IMAGE: rocm/atom-dev:latest
GITHUB_REPO_URL: ${{ github.event.pull_request.head.repo.clone_url || 'https://github.com/ROCm/ATOM.git' }}
GITHUB_COMMIT_SHA: ${{ github.event.pull_request.head.sha || github.event.head_commit.id || github.sha }}
# workflow_dispatch: inputs.aiter_branch; otherwise main (matches previous default-branch shallow clone)
AITER_GIT_REF: ${{ github.event_name == 'workflow_dispatch' && inputs.aiter_branch || 'main' }}
GITHUB_REPO_URL: ${{ inputs.source_repo_url || format('https://github.com/{0}.git', inputs.checkout_repository) }}
GITHUB_COMMIT_SHA: ${{ inputs.source_commit_sha || inputs.checkout_ref }}
AITER_GIT_REF: ${{ inputs.aiter_branch }}
Comment on lines +73 to +75
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new run_precheck / source_commit_sha inputs allow callers to enable the pre-checkin gate, but GITHUB_COMMIT_SHA currently falls back to inputs.checkout_ref (which may be a branch/ref string). Since the pre-checkin artifact naming is SHA-based (checks-signal-<sha>), enabling run_precheck with a non-SHA checkout_ref will deterministically fail after retries. Consider validating that the commit input is a full SHA (or resolve the checked-out ref to a SHA via git rev-parse) before running the signal download to fail fast with a clear error.

Copilot uses AI. Check for mistakes.

jobs:
check-signal:
if: ${{ !github.event.pull_request || github.event.pull_request.draft == false }}
if: ${{ !inputs.is_draft_pr && inputs.run_precheck }}
name: Check Pre Checkin Signal
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
steps:
- name: Checkout ATOM repo
if: ${{ github.event_name != 'workflow_dispatch' }}
uses: actions/checkout@v6
with:
repository: ${{ inputs.checkout_repository }}
ref: ${{ inputs.checkout_ref }}
token: ${{ secrets.github_token || github.token }}

- name: Download and check pre-checkin signal
if: ${{ github.event_name != 'workflow_dispatch' }}
run: bash ./.github/scripts/check_signal.sh
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_SHA: ${{ github.sha }}
GITHUB_TOKEN: ${{ secrets.github_token || github.token }}
GITHUB_REPOSITORY: ${{ inputs.checkout_repository }}
GITHUB_SHA: ${{ inputs.signal_commit_sha || inputs.checkout_ref }}

download_aiter_wheel:
if: ${{ needs.check-signal.result == 'success' && (!github.event.pull_request || github.event.pull_request.draft == false) }}
if: ${{ !inputs.is_draft_pr && (!inputs.run_precheck || needs.check-signal.result == 'success') }}
needs: [check-signal]
name: Download aiter wheel
runs-on: ubuntu-latest
Expand All @@ -67,7 +110,7 @@ jobs:

S3_MAIN_MANIFEST_URL="https://rocm.frameworks-nightlies.amd.com/whl-staging/gfx942-gfx950/main/latest.json"
API_URL="https://api.github.com"
AUTH_HEADER="Authorization: token ${{ secrets.GITHUB_TOKEN }}"
AUTH_HEADER="Authorization: token ${{ secrets.github_token || github.token }}"
AITER_TEST_WORKFLOW_ID=179476100

download_from_s3_manifest() {
Expand Down Expand Up @@ -162,28 +205,32 @@ jobs:
retention-days: 1

load-test-models:
if: ${{ !inputs.is_draft_pr }}
name: Load test model configs
runs-on: ubuntu-latest
outputs:
models_json: ${{ steps.load.outputs.models_json }}
steps:
- uses: actions/checkout@v6
with:
repository: ${{ inputs.checkout_repository }}
ref: ${{ inputs.checkout_ref }}
token: ${{ secrets.github_token || github.token }}
- id: load
env:
EVENT_NAME: ${{ github.event_name }}
MODEL_SCOPE: ${{ inputs.model_scope }}
run: |
python3 << 'PY'
import json, os
event = os.environ["EVENT_NAME"]
# pr → pr models only; push to main → pr+main; schedule/dispatch → all
level_map = {"schedule": "nightly", "workflow_dispatch": "nightly", "push": "main"}
current = level_map.get(event, "pr")
allowed = {"pr": {"pr"}, "main": {"pr", "main"}, "nightly": {"pr", "main", "nightly"}}[current]
current = os.environ["MODEL_SCOPE"]
allowed = {"pr": {"pr"}, "main": {"pr", "main"}, "nightly": {"pr", "main", "nightly"}}.get(current)
if allowed is None:
raise SystemExit(f"Unsupported model_scope: {current}")
models = json.load(open(".github/benchmark/models_accuracy.json", encoding="utf-8"))
filtered = [m for m in models if m.get("test_level", "nightly") in allowed]
with open(os.environ["GITHUB_OUTPUT"], "a") as f:
f.write(f"models_json={json.dumps(filtered)}\n")
print(f"Event={event} level={current}: {len(filtered)}/{len(models)} models")
print(f"Scope={current}: {len(filtered)}/{len(models)} models")
print(f"{'Model':<45} {'Level':<10} {'Runner'}")
print("-" * 80)
for m in models:
Expand All @@ -198,7 +245,7 @@ jobs:
fail-fast: false
matrix:
include: ${{ fromJson(needs.load-test-models.outputs.models_json) }}
if: ${{ !github.event.pull_request || github.event.pull_request.draft == false }}
if: ${{ !inputs.is_draft_pr }}
runs-on: ${{ matrix.runner }}

env:
Expand Down Expand Up @@ -229,17 +276,28 @@ jobs:

- name: Checkout ATOM repo
uses: actions/checkout@v6
with:
repository: ${{ inputs.checkout_repository }}
ref: ${{ inputs.checkout_ref }}
token: ${{ secrets.github_token || github.token }}

- name: Docker Login
if: ${{ !github.event.pull_request.head.repo.fork }}
if: ${{ !inputs.is_fork_pr }}
env:
DOCKER_USERNAME: ${{ secrets.docker_username }}
DOCKER_PASSWORD: ${{ secrets.docker_password }}
run: |
echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u ${{ secrets.DOCKER_USERNAME }} --password-stdin
if [ -z "${DOCKER_USERNAME:-}" ] || [ -z "${DOCKER_PASSWORD:-}" ]; then
echo "Docker credentials not provided; skipping docker login."
exit 0
fi
echo "${DOCKER_PASSWORD}" | docker login -u "${DOCKER_USERNAME}" --password-stdin

- name: Generate Dockerfile for forked repo
if: ${{ github.event.pull_request.head.repo.fork }}
if: ${{ inputs.is_fork_pr }}
run: |
cat <<EOF > Dockerfile.mod
FROM ${{ env.ATOM_BASE_NIGHTLY_IMAGE }}
FROM ${{ env.ATOM_BASE_IMAGE }}
RUN pip install -U lm-eval[api]
RUN pip show lm-eval || true
RUN pip install hf_transfer
Expand Down Expand Up @@ -274,7 +332,7 @@ jobs:

- name: Set HF token for predownload runner
if: matrix.runner == 'atom-mi355-8gpu.predownload'
run: echo "HF_TOKEN=${HF_TOKEN:-${{ secrets.AMD_HF_TOKEN }}}" >> "$GITHUB_ENV"
run: echo "HF_TOKEN=${HF_TOKEN:-${{ secrets.amd_hf_token }}}" >> "$GITHUB_ENV"

- name: Start CI container
run: |
Expand Down Expand Up @@ -566,10 +624,17 @@ jobs:
accuracy-dashboard:
name: Update accuracy dashboard
needs: [atom-test]
if: always() && github.ref == 'refs/heads/main' && (github.event_name == 'push' || github.event_name == 'schedule')
if: ${{ always() && inputs.publish_dashboard }}
runs-on: ubuntu-latest
Copy link

Copilot AI Apr 14, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

publish_dashboard can cause this job to push to gh-pages, but accuracy-dashboard does not declare permissions: contents: write (and likely actions: read for artifact download). In reusable workflows, callers often set the default GITHUB_TOKEN permissions to read-only, which will make the dashboard publish fail even when publish_dashboard is true. Add an explicit permissions block on this job (or document/require the caller to grant these permissions) so reuse is reliable.

Suggested change
runs-on: ubuntu-latest
runs-on: ubuntu-latest
permissions:
contents: write
actions: read

Copilot uses AI. Check for mistakes.
permissions:
contents: write
actions: read
steps:
- uses: actions/checkout@v6
with:
repository: ${{ inputs.checkout_repository }}
ref: ${{ inputs.checkout_ref }}
token: ${{ secrets.github_token || github.token }}

- uses: actions/setup-python@v6
with:
Expand Down Expand Up @@ -606,5 +671,5 @@ jobs:
benchmark-data-dir-path: benchmark-dashboard
auto-push: true
max-items-in-chart: 90
github-token: ${{ secrets.GITHUB_TOKEN }}
github-token: ${{ secrets.github_token || github.token }}

Loading