diff --git a/CLAUDE.md b/CLAUDE.md index 2597b28a..58d8bbd7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -111,7 +111,7 @@ with TOGSimulator(config_path=...): - **Docker (recommended):** `docker run -it --ipc=host --name torchsim -w /workspace/PyTorchSim ghcr.io/psal-postech/torchsim-ci:v1.0.1 bash` - **TOGSim from source:** `cd TOGSim && mkdir -p build && cd build && conan install .. --build=missing && cmake .. && make -j$(nproc)` - **PyTorchSimDevice (Python package):** `cd PyTorchSimDevice && python -m pip install --no-build-isolation -e .` -- **gem5 / LLVM+MLIR / Spike from source:** `bash scripts/build_from_source.sh` (clones to `/workspace/{gem5,llvm-project,riscv-isa-sim}`) +- **gem5 / LLVM+MLIR / Spike from source:** `bash scripts/build_from_source.sh` (clones to `/workspace/{gem5,llvm-project,riscv-isa-sim}` at the tags pinned in `thirdparty/github-releases.json`, same manifest as the CI docker image). Conan deps for TOGSim: `boost/1.79.0`, `robin-hood-hashing/3.11.5`, `spdlog/1.11.0`, `yaml-cpp/0.8.0`. @@ -137,5 +137,6 @@ Conan deps for TOGSim: `boost/1.79.0`, `robin-hood-hashing/3.11.5`, `spdlog/1.11 ## Git workflow (per CONTRIBUTING.md) -- Fork → branch (`feature/`) → PR against **`develop`**, not `main`. +- Fork, branch (`feature/`), PR against `develop`, not `main`. - Commit prefix style observed: `[Frontend] ...`, `[TOGSim] ...`, etc. +- Commit messages: plain text only. No Markdown formatting (no backticks, bold, bullet lists, headings). Avoid Unicode where ASCII works (use `->` not arrows, `--` not em-dashes, straight quotes). diff --git a/README.md b/README.md index 042d3185..800f4761 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,7 @@ This script builds [Gem5](https://github.com/PSAL-POSTECH/gem5.git), [LLVM](http ```bash bash scripts/build_from_source.sh ``` +The script clones each dep at the tag pinned in [`thirdparty/github-releases.json`](thirdparty/github-releases.json), the same manifest the CI docker image uses, so a from-source build matches the docker env. ### Run Examples The `tests` directory contains several AI workload examples. ```bash diff --git a/scripts/build_from_source.sh b/scripts/build_from_source.sh index fb9e82e3..4e7ff604 100644 --- a/scripts/build_from_source.sh +++ b/scripts/build_from_source.sh @@ -1,22 +1,60 @@ -#!/bin/bash +#!/usr/bin/env bash +# Build Gem5 / LLVM+MLIR / Spike from source. +# +# Versions are pinned in thirdparty/github-releases.json - the same manifest +# the CI docker image (ghcr.io/psal-postech/torchsim-ci) is built against. +# Cloning untagged HEADs has caused mlir-opt option-name drift in the past +# (e.g. test-tile-operation-graph's `sample-mode` <-> `tls_mode` rename), so +# always honor the pinned release_tag for a known-good Python<->mlir-opt pair. +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +MANIFEST="${ROOT}/thirdparty/github-releases.json" home="/workspace" -cd $home + +if [ ! -f "$MANIFEST" ]; then + echo "error: pin manifest not found at $MANIFEST" >&2 + exit 1 +fi +if ! command -v jq >/dev/null 2>&1; then + echo "jq not found, installing..." + apt -y update && apt -y install jq +fi + +read_pin() { + # $1 = key (gem5 / llvm_project / spike), echoes " " + jq -r --arg k "$1" '.[$k] | "\(.repository) \(.release_tag)"' "$MANIFEST" +} + +read GEM5_REPO GEM5_TAG <<< "$(read_pin gem5)" +read LLVM_REPO LLVM_TAG <<< "$(read_pin llvm_project)" +read SPIKE_REPO SPIKE_TAG <<< "$(read_pin spike)" + +echo "Building from source using pins in $MANIFEST:" +echo " gem5 = ${GEM5_REPO} @ ${GEM5_TAG}" +echo " llvm = ${LLVM_REPO} @ ${LLVM_TAG}" +echo " spike = ${SPIKE_REPO} @ ${SPIKE_TAG}" + +cd "$home" # Gem5 apt -y update && apt -y upgrade && apt -y install scons -git clone https://github.com/PSAL-POSTECH/gem5.git -cd gem5 && scons build/RISCV/gem5.opt -j $(nproc) -export GEM5_PATH=$home/gem5/build/RISCV/gem5.opt -cd $home - -# LLVM -git clone https://github.com/PSAL-POSTECH/llvm-project.git -cd llvm-project && mkdir build && cd build && \ - cmake -DLLVM_ENABLE_PROJECTS=mlir -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/riscv-llvm -DLLVM_TARGETS_TO_BUILD=RISCV -G "Unix Makefiles" ../llvm && \ +git clone --depth 1 --branch "$GEM5_TAG" "https://github.com/${GEM5_REPO}.git" +cd gem5 && scons build/RISCV/gem5.opt -j "$(nproc)" +export GEM5_PATH="$home/gem5/build/RISCV/gem5.opt" +cd "$home" + +# LLVM + MLIR (RISCV target) +git clone --depth 1 --branch "$LLVM_TAG" "https://github.com/${LLVM_REPO}.git" +cd llvm-project && mkdir -p build && cd build && \ + cmake -DLLVM_ENABLE_PROJECTS=mlir -DCMAKE_BUILD_TYPE=Release \ + -DCMAKE_INSTALL_PREFIX=/riscv-llvm -DLLVM_TARGETS_TO_BUILD=RISCV \ + -G "Unix Makefiles" ../llvm && \ make -j && make install -cd $home +cd "$home" # Spike Simulator -git clone https://github.com/PSAL-POSTECH/riscv-isa-sim.git --branch TorchSim && cd riscv-isa-sim && mkdir build && cd build && \ - ../configure --prefix=$RISCV && make -j && make install -cd $home \ No newline at end of file +git clone --depth 1 --branch "$SPIKE_TAG" "https://github.com/${SPIKE_REPO}.git" +cd riscv-isa-sim && mkdir -p build && cd build && \ + ../configure --prefix="$RISCV" && make -j && make install +cd "$home"