Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion agents/conductors/hygiene/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ kinds, which is what makes its count comparable (or not):
| `noise` | none — needs a pytest + workspace-script run (**advisory**) | `/cli_noise_clean` (Heart) |
| `deps` | capped/pinned specifiers in library `pyproject.toml` (**surface**) | `/dep_audit` (Heart, hits PyPI) |
| `docs` | `docs/api/*.rst` + `currentmodule` counts across the 3 doc repos (**surface**) | `/audit_docs` (Heart, imports) |
| `crlf` | `.py` files with CRLF line endings across the managed repos (**debris**) | `/refactor` (`dos2unix` / `sed`) |
| `crlf` | executable scripts (`.sh` + shebang-`755` `.py`) with CRLF — the shebang breaks on Linux/HPC (**debris**, the ranked count); library `.py` CRLF is reported separately as *cosmetic* (Python reads it fine — don't mass-normalise) | `/refactor` + `.gitattributes eol=lf` |
| `config` | library `config/*.yaml` keys missing from the matching workspace config — recursive diff (**surface**) | `/refactor` (mirror keys) |
| `artifacts` | tracked files that look like leaked run outputs / stray data (under `output/`, or data-ext outside fixtures) (**debris**) | `/repo_cleanup` (gitignore + `git rm --cached`) |
| *(default)* | all of the above (**perf timing deferred** — it spawns real imports) | a ranked `HygieneDecision` worklist — recommends the highest-count debris mode (`tidy`/`crlf`/`artifacts`), then `hygiene perf`, then the periodic surface audits |
Expand Down
31 changes: 22 additions & 9 deletions agents/conductors/hygiene/hygiene.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
# hygiene.sh noise # CLI-noise route -> /cli_noise_clean
# hygiene.sh deps # dependency-cap pre-scan -> /dep_audit
# hygiene.sh docs # API-docs pre-scan -> /audit_docs
# hygiene.sh crlf # .py files with CRLF line endings -> /refactor
# hygiene.sh crlf # executable scripts w/ CRLF break on HPC (+ cosmetic .py) -> /refactor
# hygiene.sh config # library config keys missing downstream -> /refactor
# hygiene.sh artifacts # tracked leaked outputs/data -> /repo_cleanup
# hygiene.sh <mode> --json # machine-readable HygieneDecision
Expand Down Expand Up @@ -131,18 +131,31 @@ prescan_docs() {
echo "${cm}|${rst} api .rst files, ${cm} currentmodule directives across ${#DOC_REPOS[@]} repos"
}

# crlf: .py files with CRLF line endings across the managed repos (a documented
# recurring gotcha — CRLF files produce 10x diffs). Fix: sed -i 's/\r$//' / dos2unix.
# crlf: CRLF line endings, split by severity. The count that MATTERS is
# executable scripts (`.sh` + shebang-executable `.py`, mode 755): a CRLF shebang
# (`#!/bin/bash\r`) breaks execution on Linux/HPC ("bad interpreter"). Library
# `.py` CRLF is COSMETIC (Python reads CRLF fine) — reported separately, not
# ranked, since mass-normalising it is a big diff for zero functional gain
# (the real fix there is `.gitattributes * text=auto`, going forward).
prescan_crlf() {
local total=0 detail="" repo dir n
for repo in "${LIB_REPOS[@]}" "${ORG_REPOS[@]}"; do
local scripts=0 cosmetic=0 sdetail="" repo dir sh_n exe_list exe_n py_n
for repo in "${LIB_REPOS[@]}" "${ORG_REPOS[@]}" autolens_workspace autogalaxy_workspace autofit_workspace; do
dir="$ROOT/$repo"
[[ -d "$dir/.git" || -f "$dir/.git" ]] || continue
n=$(git -C "$dir" grep -Il $'\r$' -- '*.py' 2>/dev/null | wc -l | tr -d ' ')
total=$((total + n))
[[ "$n" -gt 0 ]] && detail+="${repo}:${n} "
# .sh with CRLF (all shell scripts break)
sh_n=$(git -C "$dir" grep -Il $'\r$' -- '*.sh' 2>/dev/null | wc -l | tr -d ' ')
# executable .py (mode 755 — run directly, so a CRLF shebang breaks)
exe_n=0
exe_list=$(git -C "$dir" ls-files --stage -- '*.py' 2>/dev/null | awk '$1 ~ /755$/ {print $4}')
[[ -n "$exe_list" ]] && exe_n=$(git -C "$dir" grep -Il $'\r$' -- $exe_list 2>/dev/null | wc -l | tr -d ' ')
local repo_scripts=$((sh_n + exe_n))
scripts=$((scripts + repo_scripts))
[[ "$repo_scripts" -gt 0 ]] && sdetail+="${repo}:${repo_scripts} "
# cosmetic: library .py with CRLF (informational)
py_n=$(git -C "$dir" grep -Il $'\r$' -- '*.py' 2>/dev/null | wc -l | tr -d ' ')
cosmetic=$((cosmetic + py_n))
done
echo "${total}|${total} .py files with CRLF line endings: ${detail}(fix: dos2unix / sed -i 's/\\r\$//')"
echo "${scripts}|${scripts} executable scripts w/ CRLF (BREAK on HPC — normalise + add .gitattributes eol=lf): ${sdetail}; ${cosmetic} library .py w/ CRLF (cosmetic — leave, or '* text=auto' going forward)"
}

# artifacts: tracked files that look like leaked generated outputs — anything
Expand Down
5 changes: 3 additions & 2 deletions skills/hygiene/hygiene.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ Shared routing context: `PyAutoBrain/skills/COMMANDS.md`.
perf's import timing is deferred there). This is a **dry run** — each mode
does a cheap read-only pre-scan and emits a `HygieneDecision` naming the
skill to run for the full audit. Nothing is executed or mutated. (`crlf` =
CRLF `.py` files; `config` = library→workspace config-key drift; `artifacts`
= tracked leaked outputs/data.)
executable scripts w/ CRLF that break on HPC — library `.py` CRLF reported as
cosmetic; `config` = library→workspace config-key drift; `artifacts` =
tracked leaked outputs/data.)
2. Execute the emitted plan: run the named delegate — `/repo_cleanup` (git
debris), `/cli_noise_clean`, `/dep_audit`, `/audit_docs` — for the full audit,
or for `perf` route slow imports/functions to `/refactor` / `/bug` (JAX-adapt
Expand Down