Local validation and production configuration for running PRISM as a BASE challenge under Docker Compose (long-lived combined challenge service). Swarm is not a supported install path.
git clone https://github.com/BaseIntelligence/prism.git
cd prism
uv sync --frozen --extra devPRISM depends on the immutable Base public wheel v3.1.2:
https://github.com/BaseIntelligence/base/releases/download/v3.1.2/base-3.1.2-py3-none-any.whl
#sha256=3a61c2d3a343ed6de55e80215486e3de0c9639276443d08f2ed316bc807f2ff0
(see pyproject.toml). There is no LLM gateway dependency in this pin.
.venv/bin/ruff check src
.venv/bin/mypy src/prism_challenge
.venv/bin/python -m pytest tests -qGPU re-execution, HuggingFace publication, and external provider calls are mocked in tests. The LLM gateway is not part of the test or deploy path.
PRISM_DATABASE_URL=sqlite+aiosqlite:////data/prism.sqlite3
PRISM_SHARED_TOKEN_FILE=/run/secrets/base/challenge_token
PRISM_EXECUTION_BACKEND=base_gpuThe shared token must match the token configured in the BASE master for this challenge. Prefer secret files over inline tokens.
Preferred production layout is the BASE master Compose project with one long-lived PRISM service in combined mode: the API process drains the eval queue in-process (same worker/DB as the HTTP app). Do not deploy Swarm stacks or launch ephemeral evaluator jobs from application code.
Typical challenge container settings for combined mode:
# Enable API + in-process queue drain in one long-lived service
# (exact env name follows PrismSettings; set in the Compose challenge service)
PRISM_COMBINED_MODE=trueDigest-pin the PRISM image via the master Compose install / watcher path (PRISM_IMAGE_REPOSITORY +
PRISM_IMAGE_DIGEST). Mutable latest Swarm service mutation is unsupported.
When evaluation uses the BASE Docker broker with the CI-published evaluator image:
PRISM_DOCKER_ENABLED=true
PRISM_DOCKER_BACKEND=broker
PRISM_DOCKER_BROKER_URL=http://base-docker-broker:8082
PRISM_DOCKER_BROKER_TOKEN_FILE=/run/secrets/base/challenge_token
PRISM_BASE_EVAL_IMAGE=ghcr.io/baseintelligence/prism-evaluator@sha256:<pinned>
PRISM_BASE_EVAL_GPU_COUNT=1
PRISM_DOCKER_NETWORK=noneThe scored run is single-node (torchrun --standalone --nnodes=1 --nproc-per-node=1). The evaluator
image must ship sentencepiece and an offline tiktoken gpt2 cache so reference tokenizers load with no
network. Prefer digest-pinned references over floating tags.
The broker bind-mounts the locked FineWeb-Edu data read-only into the eval container (network=none):
PRISM_BASE_EVAL_DATA_DIR=/data/fineweb-edu/train # miner-visible, read-only
PRISM_BASE_EVAL_VAL_DATA_DIR=/data/fineweb-edu/val # secret; scorer-only, never mounted into eval
PRISM_BASE_EVAL_REFERENCE_TOKENIZER_DIR=/opt/reference-tokenizersHF_HUB_OFFLINE=1 and HF_DATASETS_OFFLINE=1 are set inside the eval container. The val/test
splits are secret and must never be exposed to a miner script.
Compute-normalized scoring; wall-clock is only a layered safety cap:
PRISM_BASE_EVAL_BUDGET_SECONDS=1200 # graceful stop; score the partial stream
PRISM_BASE_EVAL_WATCHDOG_GRACE_SECONDS=120 # hard watchdog above the graceful budget
PRISM_BASE_EVAL_TIMEOUT_SECONDS=1800 # outer docker/broker backstop
PRISM_BASE_EVAL_ARTIFACTS_QUOTA_BYTES=2147483648Admission is challenge-owned and deterministic. Do not set gateway URL/token or LLM review settings; residual keys fail closed at load:
# Removed — do not configure
# PRISM_LLM_REVIEW_ENABLED
# PRISM_LLM_GATEWAY_URL
# PRISM_GATEWAY_TOKEN_FILE
# BASE_LLM_GATEWAY_URL
# BASE_GATEWAY_TOKEN
Pipeline order before GPU:
- AST sandbox hard-blocks over both scripts.
- Forced-seed parameter cap (150M).
- Multi-GPU static contract and single-node bound.
- Source similarity + duplicate policy (quarantine band → terminal reject, never held).
PRISM_DISTRIBUTED_CONTRACT_POLICY=reject # reject | flag | off
PRISM_BASE_EVAL_MAX_GPU_COUNT=8reject (default) hard-rejects a non-distributed training.py; flag advances but logs; off skips
the check.
# thresholds also live in SQL runtime config under duplicate_thresholds
PRISM_PLAGIARISM_ENABLED=trueAn exact-source-hash duplicate is rejected; a borderline-similarity quarantine is folded into a terminal rejection at ingress. There is no operator hold-resolution surface.
When master coordination is configured, PRISM periodically pushes authenticated raw hotkey weights to
the master private ingress (versioned epoch/revision, digest, idempotent). Configure master base URL
and challenge-scoped credentials via settings/secret files; never log tokens. Validators submit the
master-aggregated vector on-chain; PRISM does not call set_weights.
Worker-plane result ingest accepts only ExternalResultEnvelope from the Base SDK. Legacy bodies
without the full binding/proof schema fail closed.
TEE:
- Local cryptographic fixtures may yield
LOCAL-FIXTURE PASSunder the fail-closed verifier. That label is the only elevated local-crypto success class. It must never be rewritten, summarized, or badged asREAL-PROVIDER PASS, production mine, or live-emission authority (APIs, CLI, audit records, and lab reports). - Real Lium/Targon production PASS is blocked until contracts and digests exist. Paid Lium
provision smoke is
DEPLOY SMOKE PASS|FAILonly and independent of crypto classification. - Adapter readiness reports always expose
would_grant_real_provider_pass=falsefor Lium/Targon in this code path, even when credentials are present, inventory is reachable,lium_ready/targon_readyis flipped, or a complete-lookingprovider_contractblob is supplied. - Targon remains
future/blocked(PROVIDER_FUTURE_BLOCKED) for REAL-PROVIDER PASS. - Safe provider probes (inventory/health) never become REAL-PROVIDER PASS.
- Readiness hard gates still in force (
HARD_GATE_ITEMS, all required for any future real PASS):authoritative_evidence_endpointauthoritative_evidence_formatauthoritative_issuer_audienceauthoritative_trust_rootsfreshness_and_clock_policynonce_semanticsdigest_pinned_public_worker_imagemeasurement_policygpu_claim_policycross_binding_semanticsreal_workload_evidence_artifact
- Residual removed LLM env/keys (
PRISM_LLM_*, gateway tokens, component-agent knobs) fail closed at settings load and never fall through to scored operation.
PRISM_SHARED_TOKEN=dev-secret \
PRISM_DATABASE_URL=sqlite+aiosqlite:///./prism.sqlite3 \
.venv/bin/uvicorn prism_challenge.app:app --host 0.0.0.0 --port 8000PRISM registers as a challenge image reached by the master over the internal challenge network on
Compose. Public miner traffic goes through the BASE proxy. Use the master Compose install path
(deploy/compose/install-master.sh and related manifests), not Swarm.
Atomic rebrand deploy: internal auth headers use
X-Base-*(X-Base-Challenge-Slug,X-Base-Verified-Hotkey). PRISM accepts onlyX-Base-*, so cut over all services together; any sender left on legacyX-Platform-*fails internal auth until the cutover completes.
curl http://localhost:8000/health
curl http://localhost:8000/version
curl -H "Authorization: Bearer dev-secret" -H "X-Base-Challenge-Slug: prism" \
http://localhost:8000/internal/v1/get_weightsOperators can rank two unknown-style seed packages under Prism Official Comparison Protocol v1 without NVIDIA. The dual-family harness packages Transformer tiny-1m and pure-torch Mamba, builds comparable official score records from challenge-owned metrics, and prints a clear A-vs-B compare_official outcome. Long multi-step GPU pair trains remain DEFERRED when the host has no NVIDIA; tee notes never claim REAL-PROVIDER PASS.
Full ranking axioms: Official Comparison.
# From a Prism checkout (dev deps installed)
export UV_CACHE_DIR=/var/tmp/uv-cache
# 1) Package both families as two-script submit zips (shared outer contract)
uv run python -m prism_challenge.seed_packaging --output-dir dist/seed-packages
# 2) Run the dual-family official compare harness (CPU/fixture, no GPU train)
uv run python -m prism_challenge.evaluator.official_compare_harness \
--output-dir dist/official-compare \
--device-class fixture
# 3) Inspect the report
cat dist/official-compare/prism_compare_report.v1.jsonTargeted unit proof (no NVIDIA):
uv run pytest tests/test_official_compare_harness.py tests/test_official_comparison_scoring.py -qReport fields of interest:
| Field | Meaning |
|---|---|
ranking.winner |
a, b, or tie under held-out primary then bpb secondary |
ranking.outcome_label |
Human-readable winner label (package family id) |
side_a / side_b |
Bundle hashes, mean held-out + recomputed bpb, validity |
gpu_verification.status |
DEFERRED when nvidia-smi//dev/nvidia*/nvidia runtime absent; never claimed PASS by the harness |
tee_note |
Always orthogonal: REAL-PROVIDER PASS not claimed |
Do not treat the offline fixture winner as an automatic emission weight crown unless production leaderboard scoring independently agrees (leaderboard stays bpb-primary; Official Comparison invert is the lab surface only).
Additive annex on Protocol v1: scorecard_id=multimetric.v1.1. Full catalogue and polar rules: Official Comparison §14.
| Operator note | Detail |
|---|---|
| Default winner | When axes do not polar-conflict, keep v1 compare_official held-out primary then bpb secondary |
| Polar conflict | If short-gen winner ≠ long-ctx winner beyond ε (or floor veto on one side only), authoritative claim is TIE_POLAR, crown_allowed=false — publish the scorecard vector; no solitary arch crown |
| Multi-seed | Public non-provisional scorecard requires clean K≥3; K=1 is provisional only |
| Prior LAB-GPU | Prior short-ctx K=1 mamba heldout lead is provisional only; insufficient for architecture superiority |
| Suites | Validity V, short-gen, long-ctx (needle / MQAR / induction-copy / lag-NLL), sample-efficiency, memo, efficiency (VRAM / tok/s / params), stability — mark not-run honestly if a suite is missing |
| TEE / ops | REAL-PROVIDER TEE stays BLOCKED; always-terminate paid pods; no live Swarm mutate; no set_weights from compare |
Inspect scorecard fields when present:
jq '{protocol_id, scorecard_id, scorecard, ranking, real_provider_tee, honesty_note}' \
dist/official-compare/prism_compare_report.v1.jsonMAX A→Z machine dashboard on Protocol v1: schema=complete_view.v1.3, scorecard_id=multimetric.complete.v1.3. Historical multimetric.complete.v1.2 and multimetric.v1.1 annex stay valid. Full matrix, multi-axis comparison (per-axis leads including reasoning, disagreement matrix, expanded TIE_POLAR, no opaque weighted sole crown), P10_reasoning_logic synthetic probes (not GSM8K/MMLU primary; seed-scale lab comparison only, not human AGI), and non-claims: Official Comparison §16 (plus §15 for v1.2 history).
| Operator note | Detail |
|---|---|
| Document | Prefer single machine file complete_view.v1.3.json reconciling panels P0–P10 + comparison |
| Rank | Multi-axis object only; scientific axis disagreements (including short_gen vs reasoning) → TIE_POLAR / crown_allowed=false |
| Reasoning panel | P10_reasoning_logic closed-acc + forced CE + chance baselines; suite_mean shell until probe suite fills |
| Honesty | Seed-scale synthetic logic is lab/architecture comparison only; not human AGI; not emission crown |
| Suites not-run | null + reason; never invent metrics |
| TEE / ops | REAL-PROVIDER TEE BLOCKED; no live Swarm mutate; no set_weights; always-terminate paid remesure pods |
| Product module | prism_challenge.evaluator.complete_view |
jq '{schema, scorecard_id, historical_scorecard_id, comparison, real_provider_tee, non_claims, p10: .panels.P10_reasoning_logic.status}' \
complete_view.v1.3.jsonWhen real dual-family CUDA trains already completed on a remote GPU host (for example paid Lium under a matched Protocol v1 pin), rank on any CPU mission host from the challenge-owned prism_run_manifest.v2.json artifacts. This path does not require local NVIDIA and is not fixture-only synthetic ranking.
| Class | When |
|---|---|
| fixture / CPU | Synth metrics + seed packaging only; host gpu_verification.status=DEFERRED with no local NVIDIA |
| LAB-GPU (short or long) | Real CUDA trains produced manifests; host recompute via official_record_from_manifest + compare_official; report score_class=LAB-GPU |
| REAL-PROVIDER TEE | Always BLOCKED / NOT_CLAIMED on this path. Lab score success never unlocks REAL-PROVIDER PASS |
Layout expected under --lab-gpu-artifacts:
{artifacts_root}/
transformer-tiny-1m/seed-1337/prism_run_manifest.v2.json
mamba-tiny-1m/seed-1337/prism_run_manifest.v2.json
export UV_CACHE_DIR=/var/tmp/uv-cache
# Host rank of real LAB-GPU long-train artifacts (example mission evidence layout)
uv run python -m prism_challenge.evaluator.official_compare_harness \
--lab-gpu-artifacts /path/to/lium-train/artifacts/out \
--output-dir dist/official-compare-lab-gpu \
--seed 1337
# Inspect
jq '{score_class, ranking, real_provider_tee, gpu_verification}' \
dist/official-compare-lab-gpu/prism_compare_report.v1.jsonProtocol axioms still hold: held-out primary, prequential bpb secondary, wall-clock may be recorded but never ranks, miner self-report is non-authoritative. Exit code 2 with BLOCKED: means dual-family manifests were missing — do not invent scores. No HA-live Swarm mutate and no set_weights are required for ranking.
Provisional honesty for prior K=1 short-ctx LAB-GPU wins: a single-seed mamba heldout_delta lead under short context is a valid provisional lab observation under Protocol v1, not multi-seed public architecture superiority. Apply the multimetric.v1.1 scorecard (long-ctx, sample-eff, K≥3, efficiency, polar rules) before any crown language. REAL-PROVIDER TEE remains BLOCKED on this path.
Machine identity: schema=prism_train_series.v1. Full protocol: Official Comparison §17.
| Operator note | Detail |
|---|---|
| Authority | Challenge-owned series only (online CE/bpb, tokens_seen, wall, mandatory grad_norm + clip_events under the telemetry pin). Miner dashboards / self-logs are non-authoritative and never certify grade |
| Scientific miner grade | Multi-axis Official Comparison / Complete View (held-out primary + bpb secondary + polar honesty) |
| Emission leaderboard | Remains bpb-primary — series are not emission substitution |
| Rank role of series | Visibility + densify sample-eff / stability residual only — never sole primary rank over held-out/bpb |
| Fail-closed Official pin | If grading pin sets require_train_series and series is missing/empty/corrupt → Official scientific grade fail-closes (not silent PASS) |
| APIs | GET /v1/submissions/{id}/curve (public challenge routes under existing Base proxy / internal auth) returns legacy loss_curve and optional challenge-owned train_series (prism_train_series.v1) with downsample-safe multichannel points: train CE / running bpb, tokens_seen, wall_s, grad_norm, clip_event. Miner authority payloads are never returned. Chart-safe key projection strips unknown/secret fields. Frontend Architecture Lab already plots loss vs covered bytes from this route; operators may also plot cosine-style time-flow via CLI jq on train_series.points until UI surfaces grad_norm series natively. |
| TEE / ops | Series realize lab observability only; REAL-PROVIDER TEE BLOCKED; no live Swarm mutate; no set_weights from telemetry |
# Operator time-flow: loss + grad_norm vs tokens / wall (same auth as other curve reads)
curl -sS -H "Authorization: Bearer $TOKEN" -H "X-Base-Challenge-Slug: prism" \
"http://localhost:8000/internal/v1/submissions/${SUBMISSION_ID}/curve" \
| jq '{
loss_points: .loss_curve.points,
schema: .train_series.schema,
authority: .train_series.authority,
downsampled: .train_series.downsampled,
n_total: .train_series.points_total,
sample: [.train_series.points[] | {i, tokens_seen, wall_s, train_ce_nats, running_bpb, grad_norm, clip_event}][0:8],
clip_events: .train_series.aggregates.clip_events
}'
# Artifact side-car (challenge-owned only; miner dashboards non-authoritative)
jq '{schema, authority, miner_reported_ignored, n: (.points|length), grads: [.points[].grad_norm][0:5]}' \
prism_train_series.v1.jsonUI note (VAL-TELE-008): the public Architecture Lab already wires GET .../curve into the loss/bpb chart (getSubmissionCurve → loss vs covered bytes). The API now also returns train_series for grad_norm / clip time-flow under the same trust path. Operators verify multichannel series via the curl/jq path above when frontend render of grad has not yet recaptured the optional field; do not invent a parallel miner-trust chart.
Do not rank packages solely on prettier grad_norm aesthetics. Use multi-axis Official / Complete View for scientific claims; keep emission on the bpb leaderboard path.
| Symptom | Likely cause |
|---|---|
invalid internal token |
Shared token mismatch between BASE and PRISM |
| submission rejected before container | Static sandbox, two-script contract, param cap, distributed contract, or similarity/anti-cheat |
| residual LLM/gateway env rejected | Removed gateway fields still set; remove them |
| evaluation failed | Broker, image, GPU, timeout, missing locked data, or container error |
| empty weights / no push | No completed scored submissions yet; raw-weight push disabled or master URL missing |
result_envelope_invalid |
Body is not a full ExternalResultEnvelope |
| TEE elevated tier never grants | Expected until real-provider contracts land; only local fixtures can PASS |
missing_locked_data |
The read-only FineWeb-Edu train mount is absent or empty on the GPU node |
| Offline compare says GPU DEFERRED | Expected without nvidia-smi on the fixture/CPU path; use fixture metrics or list CUDA as metadata only |
| LAB-GPU report still fixture | Forgot --lab-gpu-artifacts; fixture path is the default |
BLOCKED: LAB-GPU host compare |
Missing {family}/seed-*/prism_run_manifest.v2.json for one or both families |
| LAB-GPU vs REAL-PROVIDER | Lab GPU score class is scientific only; REAL-PROVIDER TEE remains BLOCKED |
| Prior mamba short-ctx win treated as “Mamba better architecture” | K=1 short-ctx LAB-GPU is provisional; require multimetric.v1.1 scorecard + K≥3; TIE_POLAR if short vs long axes disagree |
| Scorecard long-ctx / sample-eff fields missing | Suite not run yet — mark not-run/BLOCKED; do not invent metrics |
| Official grade missing train series | Pin requires prism_train_series.v1 and capture empty/miner-only — fail-closed Official grade; do not PASS on miner dashboard |
| Ranking only on grad aesthetics / wall_s | Forbidden — series residual never sole-primary; emission stays bpb-primary; Official heldout/bpb multi-axis still rule |