An "ability to learn" ML challenge — two-script submissions, locked data, challenge-owned scoring.
Overview · Miners · Validators · Architecture · Scoring · Security
PRISM is a BASE subnet challenge that measures a model's ability to learn
from scratch. Miners submit a two-script bundle — architecture.py (build_model(ctx)) and
training.py (train(ctx)) — and the challenge owns everything else: a locked FineWeb-Edu
dataset (read-only, no network) and the scoring. The miner owns the model and the training loop;
the challenge owns the data and the score.
Every scored run is re-executed under a forced random init, so the score is a prequential
(online) compression metric in bits-per-byte — the area under the from-scratch loss curve,
normalized by bytes consumed. Admission and scoring are deterministic (no LLM gateway). Raw
weights are pushed to BASE for master aggregation; validators fetch the final vector and call
set_weights under their own hotkeys.
PRISM depends on the immutable Base public wheel:
https://github.com/BaseIntelligence/base/releases/download/v3.1.2/base-3.1.2-py3-none-any.whl
#sha256=3a61c2d3a343ed6de55e80215486e3de0c9639276443d08f2ed316bc807f2ff0
(see pyproject.toml). There is no LLM gateway dependency in this pin.
flowchart LR
M[Miner two-script bundle] --> G{Static sandbox + param cap}
G -- reject --> X[[rejected]]
G --> A[Deterministic admission]
A --> V[Validator re-executes<br/>forced random init]
V --> S[Prequential bpb + held-out delta]
S --> W[Raw-weight push → BASE master]
- Submit — a signed
architecture.py+training.pybundle (a single combined module is rejected). - Static gates — AST sandbox, 150M parameter cap, single-node multi-GPU contract; any failure is terminal before GPU.
- Deterministic admission — challenge-owned checks only; the former LLM gateway hard gate is removed.
- Forced-init re-execution — one validator re-runs the loop on the locked FineWeb-Edu train split and captures the online loss itself (miner-reported numbers are ignored).
- Scoring — the challenge computes prequential bits-per-byte plus a secret held-out delta tie-breaker.
- Weights — emission splits two-tier (best architecture
0.60/ best training variant0.40); raw weights push to BASE master aggregation, then validators submit on-chain (or a fake chain in tests).
Common cheats are inert, not merely detected:
- No pretrained weights — forced random init makes smuggled weights inert; an anomalous step-0 loss zeroes the score; the container runs
network=none. - No metric gaming — the challenge recomputes the metric from the loss it captured; miner-reported numbers and manifests are ignored.
- No memorization — the secret
val/testsplits never leave the master; an excessive train-vs-held-out gap is penalized. - Deterministic — fixed seeds and a challenge-controlled data order reproduce the same score within tolerance.
PRISM includes a Prism-only, fail-closed local TEE fixture verifier for unit and contract tests. Real Lium/Targon remote attestation that would produce a production PASS is blocked until those provider readiness gates are satisfied. Local fixture verification does not imply live TEE production readiness on Lium or Targon.
PRISM can move GPU re-execution onto miner-funded workers (deployed on Lium/Targon via the BASE
base worker CLI). Validators then run verify-only plausibility checks plus probabilistic audits,
and each result carries an ExecutionProof (manifest hash + worker sr25519 signature, with optional
image-digest and attestation tiers). Gated behind worker_plane (default off). See the
worker deployment guide.
| Guide | Contents |
|---|---|
| Overview | The challenge in one page |
| Miner guide | Build and submit a two-script bundle |
| Validator guide | Run evaluation on your own broker |
| Architecture | Service design and forced-init re-execution |
| Submission format | The two-script contract and PrismContext |
| Scoring & rewards | Leaderboard prequential bits-per-byte and tie-breakers |
| Official Comparison v1 | Held-out primary / bpb secondary pair protocol (lab) + multimetric.v1.1 scorecard annex |
| Scaling | Single-node multi-GPU contract |
| Security model | Sandbox, deterministic admission, anti-cheat |
| API | Internal and public routes |
| Operators | Deploy and run under BASE Compose |
uv run ruff check .
uv run mypy
uv run pytest --cov=prism_challenge --cov-fail-under=80GPU re-execution, HuggingFace publication, and external provider calls are mocked in tests; real GPU and provider keys are wired only at deploy. The LLM gateway is not part of the test or deploy path.
Apache-2.0