Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions Makefile.toml
Original file line number Diff line number Diff line change
Expand Up @@ -60,12 +60,7 @@ dependencies = [
[tasks.check-site-content]
workspace = false
script = [
"python3 scripts/github/validate_change_bundle.py artifacts/github/bundles",
"python3 scripts/github/validate_upstream_review.py artifacts/github/review-queue",
"python3 scripts/github/validate_upstream_review.py artifacts/github/reviews",
"python3 scripts/github/test_social_post_contract.py",
"python3 scripts/github/validate_social_post.py artifacts/social/x",
"python3 scripts/github/validate_signal_entry.py site/src/content/signals",
"cargo run -p decodex --bin decodex -- radar validate",
]

[tasks.check-site-types]
Expand Down
15 changes: 8 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,10 @@ runtime.
- `apps/decodex-app/` owns the native macOS app that manages Decodex
Codex accounts through the bundled Rust app helper.
- `site/` owns the Astro static site and checked-in public content.
- `apps/decodex/src/radar.rs` owns Rust Radar queue, release-delta, and validation
commands.
- `scripts/github/` owns deterministic GitHub bundle, render, validation, backfill,
ledger import, and analysis-support scripts.
- `apps/decodex/src/radar.rs` owns Rust Radar queue, release-delta, bundle, render,
validation, backfill, and ledger commands.
- `scripts/github/` owns the automation-only Codex AI analysis helper and shared
schema support for that helper.
- `artifacts/github/` owns checked-in review queues, upstream reviews, GitHub bundles,
impact records, and editorial analysis drafts.
- `artifacts/archive/` owns checked-in recovery manifests for cold Radar batches stored
Expand All @@ -79,8 +79,8 @@ runtime.

Runtime authority stays in `apps/decodex/src/`, the registered project contracts under
`~/.codex/decodex/projects/<service-id>/`, and the governing specs under `docs/spec/`.
Public site authority stays in `site/`, `scripts/github/`, `artifacts/github/`, and
the site/content specs.
Public site authority stays in `site/`, `apps/decodex/src/radar.rs`,
`artifacts/github/`, and the site/content specs.

Historical Radar trace is local by default. `decodex radar refresh-upstream-queue`
writes `.decodex/radar.sqlite3` and refreshes `upstream_review_queue/v1` so every
Expand Down Expand Up @@ -215,7 +215,8 @@ Codex automation reviews source evidence:
- `docs/spec/upstream-impact.md` records how upstream Codex changes are classified for
public signals and Control Plane follow-up work.
- `decodex radar render-signal` renders reviewed analysis drafts into site content.
- `scripts/github/validate_signal_entry.py` validates the published signal collection.
- `decodex radar validate` validates the published signal collection and checked Radar
artifact contracts.
- `decodex radar refresh-upstream-queue`, `decodex radar refresh-release-delta`,
`decodex radar bundle validate`, `decodex radar ledger ...`, `decodex radar
render-signal`, `decodex radar backfill-release-range`, and `decodex radar
Expand Down
2 changes: 1 addition & 1 deletion apps/decodex/src/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1028,7 +1028,7 @@ struct RadarBackfillReleaseRangeCommand {
/// Compare pair limit passed through only by --refresh-release-delta-first.
#[arg(long)]
refresh_pair_limit: Option<usize>,
/// Python executable for non-ported helper boundaries.
/// Python executable for the Codex AI analysis helper boundary.
#[arg(long, default_value = "python3")]
python_bin: String,
}
Expand Down
51 changes: 22 additions & 29 deletions apps/decodex/src/radar.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,6 @@ const UPSTREAM_SUBJECT_KINDS: &[&str] = &["commit", "pr"];
const GENERIC_COMMIT_TITLES: &[&str] =
&["update", "fix", "fix.", "fix tests", "fix tests.", "merge fixes", "flaky syntax"];
const CONFIG_FEATURE_CATALOG_PATH: &str = "site/src/generated/codex-config-features.json";
const BUILD_RELEASE_DELTA_SCRIPT: &str = "scripts/github/build_release_delta.py";
const RUN_CODEX_ANALYSIS_SCRIPT: &str = "scripts/github/run_codex_analysis.py";
const HIGH_VALUE_SURFACES: &[&str] = &[
"app_server_protocol",
Expand Down Expand Up @@ -415,7 +414,7 @@ pub(crate) struct RadarBackfillReleaseRangeRequest {
pub(crate) refresh_preview_limit: Option<usize>,
/// Compare-pair limit passed through only when refreshing first.
pub(crate) refresh_pair_limit: Option<usize>,
/// Python executable used for non-ported helper boundaries and the AI boundary.
/// Python executable used for the Codex AI analysis helper boundary.
pub(crate) python_bin: String,
}

Expand Down Expand Up @@ -1252,7 +1251,7 @@ pub(crate) fn backfill_release_range(
}

validate(&RadarValidateRequest { paths: vec![resolve_against(&root, &request.signals_dir)] })?;
run_build_release_delta(&root, request, &request.release_delta, false)?;
run_refresh_release_delta(request, &request.release_delta, false)?;

Ok(report)
}
Expand Down Expand Up @@ -2814,7 +2813,7 @@ fn prepare_release_delta_path(

let release_delta = temp_root.join("release-delta.json");

run_build_release_delta(root, request, &release_delta, true)?;
run_refresh_release_delta(request, &release_delta, true)?;

Ok(PreparedReleaseDelta { path: release_delta, cleanup_dir: Some(temp_root) })
}
Expand Down Expand Up @@ -2864,40 +2863,34 @@ fn run_codex_analysis(
run_helper(command, RUN_CODEX_ANALYSIS_SCRIPT)
}

fn run_build_release_delta(
root: &Path,
fn run_refresh_release_delta(
request: &RadarBackfillReleaseRangeRequest,
out: &Path,
include_refresh_limits: bool,
) -> crate::prelude::Result<()> {
let mut command = helper_command(root, request, BUILD_RELEASE_DELTA_SCRIPT);

command.args([
"--repo",
request.repo.as_str(),
"--signals-dir",
&path_arg(root, &request.signals_dir),
"--out",
&path_arg(root, out),
]);

if let Some(token_env) = &request.token_env {
command.args(["--token-env", token_env]);
}
let mut refresh_request = RadarRefreshReleaseDeltaRequest {
repo: request.repo.clone(),
signals_dir: request.signals_dir.clone(),
out: out.to_path_buf(),
token_env: request.token_env.clone(),
..RadarRefreshReleaseDeltaRequest::default()
};

if include_refresh_limits {
push_optional_limit(&mut command, "--stable-limit", request.refresh_stable_limit);
push_optional_limit(&mut command, "--preview-limit", request.refresh_preview_limit);
push_optional_limit(&mut command, "--pair-limit", request.refresh_pair_limit);
if let Some(limit) = request.refresh_stable_limit {
refresh_request.stable_limit = limit;
}
if let Some(limit) = request.refresh_preview_limit {
refresh_request.preview_limit = limit;
}
if let Some(limit) = request.refresh_pair_limit {
refresh_request.pair_limit = limit;
}
}

run_helper(command, BUILD_RELEASE_DELTA_SCRIPT)
}
refresh_release_delta(&refresh_request)?;

fn push_optional_limit(command: &mut Command, flag: &str, value: Option<usize>) {
if let Some(value) = value {
command.args([flag, &value.to_string()]);
}
Ok(())
}

fn helper_command(
Expand Down
2 changes: 1 addition & 1 deletion dev/skills/github-signal/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,7 @@ decodex radar render-signal \
Validate the published output:

```bash
python3 scripts/github/validate_signal_entry.py site/src/content/signals
decodex radar validate site/src/content/signals
npm run build --prefix site
npm run check --prefix site
```
2 changes: 1 addition & 1 deletion dev/skills/x-post-publisher/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,5 +131,5 @@ Write `artifacts/social/x/posts/<yyyy-mm-dd>/<slug>.json` with:
Run:

```bash
python3 scripts/github/validate_social_post.py artifacts/social/x
decodex radar validate artifacts/social/x
```
8 changes: 4 additions & 4 deletions docs/reference/workspace-layout.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ should not be treated as repository source.
| `apps/decodex/` | Rust package that builds the `decodex` CLI and runtime. Runtime, orchestration, tracker integration, app-server integration, operator HTTP, and local control-plane behavior live under `apps/decodex/src/`. |
| `apps/decodex-app/` | SwiftPM macOS app for local Decodex Codex account-pool management. It talks to the bundled `decodex-app-helper`, which links the Rust account service directly, and does not own runtime scheduling or operator dashboard state. |
| `site/` | Astro static site for the public Decodex signal surface. It renders checked-in content and generated JSON from `site/src/content/`; it is not backed by a live Decodex daemon. |
| `scripts/github/` | Deterministic GitHub collection, normalization, render, validation, and sync scripts for public signal content. |
| `scripts/github/` | Automation-only Codex AI analysis helper and shared schema support for that helper. Deterministic Radar commands live in the Rust CLI. |
| `scripts/config/` | Repository automation scripts for config-derived artifacts. |
| `artifacts/github/` | Checked-in GitHub change bundles and editorial analysis drafts used by the public signal pipeline. |
| `artifacts/archive/` | Checked-in manifests for cold Radar archive batches stored as GitHub Release assets. |
Expand Down Expand Up @@ -76,7 +76,7 @@ Those runtime and operator surfaces stay in `apps/decodex/` and `docs/spec/`.

## GitHub signal tooling

`scripts/github/` owns deterministic content scripts. `decodex radar
`apps/decodex/src/radar.rs` owns deterministic Radar commands. `decodex radar
refresh-upstream-queue` is the continuous Radar entrypoint: it scans recent upstream
commits, resolves them back to PRs when possible, records local ledger state, and
writes an `upstream_review_queue/v1` artifact for Codex automation. It does not run
Expand Down Expand Up @@ -154,8 +154,8 @@ tracker routing, and policy.
- Runtime authority stays in `apps/decodex/src/`, the registered project contract under
`~/.codex/decodex/projects/<service-id>/`, and the governing specs under
`docs/spec/`.
- Public site authority stays in `site/`, `scripts/github/`, `artifacts/github/`, and
the site/content specs.
- Public site authority stays in `site/`, `apps/decodex/src/radar.rs`,
`artifacts/github/`, and the site/content specs.
- Reusable agent-facing Decodex usage instructions live under `plugins/decodex/`.
- `docs/runbook/`, `docs/reference/`, and `docs/decisions/` must not override runtime or
workflow authority.
Expand Down
4 changes: 2 additions & 2 deletions docs/runbook/local-github-signal-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ decodex radar render-signal \
Validate the published signal entries and the site collection:

```bash
python3 scripts/github/validate_signal_entry.py site/src/content/signals
decodex radar validate site/src/content/signals
npm run build --prefix site
npm run check --prefix site
cargo make decodex-checks
Expand Down Expand Up @@ -118,7 +118,7 @@ decodex radar backfill-release-range \
Use release-range backfill to fill gaps in the accumulated commit/PR analysis before a
release or prerelease summary. It should supplement continuous commit tracking, not
replace it. Execute mode is still a Codex automation or local operator path: Rust
selects the release-window gaps and sequences deterministic helper boundaries, while
selects the release-window gaps and sequences deterministic Radar commands, while
`scripts/github/run_codex_analysis.py` remains the read-only Codex AI helper that
creates validated `analysis_draft` artifacts.

Expand Down
2 changes: 1 addition & 1 deletion docs/runbook/social-publishing-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ for technical claims.
- Run:

```bash
python3 scripts/github/validate_social_post.py artifacts/social/x
decodex radar validate artifacts/social/x
```

## Mode Guidance
Expand Down
6 changes: 3 additions & 3 deletions docs/spec/github-change-bundle.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Purpose: Define the normalized GitHub input bundle that feeds Decodex signal ana
Status: normative

Read this when:
- You are writing GitHub collection or normalization scripts.
- You are changing `decodex radar bundle build` or bundle normalization behavior.
- You are deciding what data Codex should read before drafting a signal.
- You are validating whether a bundle contains enough context for GitHub-first analysis.

Expand All @@ -25,8 +25,8 @@ decodex radar bundle build --repo openai/codex --pr 15222 --out artifacts/github
decodex radar bundle validate artifacts/github/bundles/openai-codex-pr-15222.json
```

The legacy Python scripts remain shared migration contracts until the final cleanup
issue removes them.
The Rust `decodex radar bundle ...` surface is the single active deterministic bundle
command path.

Defines:
- The canonical `github_change_bundle/v1` shape.
Expand Down
11 changes: 4 additions & 7 deletions docs/spec/radar-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,8 @@ traceable without putting every raw or low-value artifact into Git.
Status: normative

Read this when:
- You are changing `decodex radar refresh-upstream-queue` or
`scripts/github/sync_upstream_radar.py`.
- You are changing `decodex radar refresh-upstream-queue`.
- You are changing `decodex radar ledger ...`.
- You are changing `scripts/github/radar_ledger.py`.
- You are importing existing GitHub bundles, analysis drafts, or signal entries into
historical Radar state.
- You need to decide what belongs in local history instead of checked-in public
Expand Down Expand Up @@ -42,10 +40,9 @@ but it is the preferred place for high-frequency trace and skip history.
## Schema

The schema is created by `decodex radar refresh-upstream-queue` and
`decodex radar ledger bootstrap`. The legacy `scripts/github/radar_ledger.py`
entrypoint remains available during migration, but the Rust
`decodex radar ledger ...` surface owns the command path for new ledger bootstrap,
ingest, ingest-existing, artifact-link, and summary operations.
`decodex radar ledger bootstrap`. The Rust `decodex radar ledger ...` surface owns the
command path for ledger bootstrap, ingest, ingest-existing, artifact-link, and summary
operations.

Required tables:

Expand Down
11 changes: 8 additions & 3 deletions scripts/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
# Scripts Root

This directory contains executable repository automation.
This directory contains executable repository automation helpers.

- `scripts/github/` owns deterministic GitHub upstream review queue, release-delta,
render, sync, and validation scripts.
- `scripts/github/` owns the automation-only Codex AI analysis helper and shared
schema support used by that helper.
- `scripts/config/` owns config-derived artifact synchronization scripts.

Checked-in data produced or consumed by scripts belongs outside this directory. GitHub
review queues, upstream reviews, bundles, impact records, and analysis drafts live
under `artifacts/github/`.

Durable deterministic Radar workflows are owned by the Rust CLI. Use
`decodex radar ...` for queue refresh, release-delta refresh, bundle build/validation,
ledger maintenance, signal rendering, release-window backfill, and Radar artifact
validation.
65 changes: 30 additions & 35 deletions scripts/github/README.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# GitHub Scripts
# GitHub Script Helpers

This directory owns deterministic GitHub-first Decodex scripts.
This directory owns the remaining Python helper boundary for GitHub-backed Radar
analysis. Durable deterministic Radar workflows live in the Rust CLI.

Current scripts:
Current helper:

- `build_change_bundle.py`
- `build_release_delta.py`
- `backfill_release_range.py`
- `radar_ledger.py`
- `run_codex_analysis.py`
- `sync_upstream_radar.py`
- `validate_change_bundle.py`
- `validate_upstream_review.py`
- `validate_social_post.py`
- `render_signal_entry.py`
- `validate_signal_entry.py`
- `run_codex_analysis.py` invokes Codex in a read-only session and writes a validated
`analysis_draft` artifact. It is the explicit AI boundary and is not a GitHub
Actions entrypoint.

Shared support:

- `contracts.py` supports the AI helper validation path.
- `analysis_draft.schema.json` is the Codex output schema for that helper.
- The remaining schema JSON files are checked contract references. They do not define
the operator command path.

Rust CLI entrypoints:

Expand All @@ -24,29 +24,24 @@ Rust CLI entrypoints:
`artifacts/github/review-queue/openai-codex-latest.json`.
- `decodex radar refresh-release-delta` refreshes
`site/src/content/release-deltas/openai-codex-latest.json`.
- `decodex radar bundle build` replaces deterministic `build_change_bundle.py` bundle
generation for PR-first and commit-only inputs.
- `decodex radar bundle validate` replaces deterministic `validate_change_bundle.py`
bundle validation.
- `decodex radar ledger ...` replaces `radar_ledger.py` bootstrap, ingest,
ingest-existing, artifact-link, and summary operations.
- `decodex radar bundle build` builds deterministic bundles for PR-first and
commit-only inputs.
- `decodex radar bundle validate` validates deterministic bundles.
- `decodex radar ledger ...` owns bootstrap, ingest, ingest-existing, artifact-link,
and summary operations.
- `decodex radar render-signal` renders `signal_entry/v1` from a validated
`github_change_bundle/v1` plus Codex-owned `analysis_draft`.
- `decodex radar backfill-release-range` selects release-window signal gaps and can
sequence the remaining helper boundaries for local or Codex automation backfills.

The Python scripts remain compatibility and non-ported helper boundaries until cleanup
issues remove them. In particular, `run_codex_analysis.py` is the explicit Codex AI
boundary for creating `analysis_draft`; it is not a GitHub Actions entrypoint.

Current checked contracts:

- `analysis_draft.schema.json`
- `upstream_review_queue/v1` is validated by `contracts.py`
- `upstream_review.schema.json`
- `release_delta/v1` is validated by `contracts.py`
- `upstream_impact.schema.json`
- `social_post.schema.json`
- `analysis_draft.schema.json` is the Codex AI helper output schema.
- `upstream_review_queue/v1` is validated by `decodex radar validate`.
- `upstream_review.schema.json` is validated by `decodex radar validate`.
- `release_delta/v1` is validated by `decodex radar validate`.
- `upstream_impact.schema.json` is validated by `decodex radar validate`.
- `social_post.schema.json` is validated by `decodex radar validate`.

Contract ownership:

Expand All @@ -69,7 +64,7 @@ decodex radar render-signal \
--analysis artifacts/github/analysis/openai-codex-pr-22414.analysis.json \
--out site/src/content/signals/openai-codex-pr-22414.json

python3 scripts/github/validate_signal_entry.py \
decodex radar validate \
site/src/content/signals/openai-codex-pr-22414.json
```

Expand Down Expand Up @@ -117,10 +112,10 @@ decodex radar backfill-release-range \
--max-prs 3
```

These scripts stay deterministic on purpose. GitHub Actions may refresh upstream
queues, release deltas, and validation. Codex automation owns AI review of queued
subjects and may then promote source-backed conclusions into `upstream_impact/v1`,
`analysis_draft`, `decodex radar render-signal` output, or `social_post/v1`.
GitHub Actions may refresh upstream queues, release deltas, and validation through
`decodex radar ...`. Codex automation owns AI review of queued subjects and may then
promote source-backed conclusions into `upstream_impact/v1`, `analysis_draft`,
`decodex radar render-signal` output, or `social_post/v1`.

Repo-local skills under `dev/skills/` are reasoning instructions for the Codex
analysis step and for manual Radar/Publisher work. They do not introduce extra
Expand Down
Loading