feat(landing): drive organ nav + sync date from a single data source#3
Conversation
GitHub Pages hosts are lowercase; normalize the uppercase host in the ecosystem and repo-card links. https://claude.ai/code/session_017iav9RLCuHkepAqnGWYMKo
Add data/site.json as the single source of truth for the landing page's dynamic data (the organ nav URLs and the footer sync date), and scripts/sync-site.py to render it into index.html via <!-- organs:start/end --> block markers and a <!-- v:synced --> value marker. Pure stdlib (no deps); --check guards against drift; SITE_DATA / SITE_HTML env overrides. The site-sync workflow runs the tests, verifies sync on PRs, and auto-commits on push/dispatch. No visible change — rendered output is byte-identical. https://claude.ai/code/session_017iav9RLCuHkepAqnGWYMKo
Reviewer's GuideCentralizes the landing page’s dynamic organ navigation and footer sync date into a JSON data file, adds a Python sync script plus tests to regenerate/index.html from that data, and wires a GitHub Actions workflow to enforce sync and auto-commit updates, with minimal HTML markers added to index.html and Python artifacts ignored via .gitignore. Sequence diagram for site-sync GitHub Actions workflowsequenceDiagram
participant GitHub as GitHub
participant Workflow as SiteSyncWorkflow
participant PyTest as scripts_test_sync_site_py
participant SyncCheck as scripts_sync_site_py_check
participant SyncWrite as scripts_sync_site_py
participant Git as git
GitHub->>Workflow: trigger push_or_pull_request
Workflow->>PyTest: python3 scripts/test_sync_site.py
PyTest-->>Workflow: tests_pass
alt event_is_pull_request
Workflow->>SyncCheck: python3 scripts/sync-site.py --check
SyncCheck-->>Workflow: exit_status
else event_is_push_or_manual
Workflow->>SyncWrite: python3 scripts/sync-site.py
SyncWrite-->>Workflow: index_html_updated
Workflow->>Git: git diff --quiet index.html
alt index_html_changed
Workflow->>Git: git add index.html
Workflow->>Git: git commit
Workflow->>Git: git push
end
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a synchronization mechanism to dynamically inject landing page data from a single source of truth (data/site.json) into index.html using a Python script (scripts/sync-site.py), complete with unit tests. The review feedback correctly identifies potential cross-platform compatibility issues on Windows systems, where default system encodings could cause UnicodeDecodeError when reading or writing files containing non-ASCII characters. It is recommended to explicitly specify encoding="utf-8" in all file read and write operations.
| data = json.loads(data_path.read_text()) | ||
| original = html_path.read_text() |
There was a problem hiding this comment.
On Windows platforms, Path.read_text() defaults to the system's active code page (e.g., CP1252). This will raise a UnicodeDecodeError when encountering non-ASCII characters like the em-dashes (—) present in index.html and site.json. Specifying encoding="utf-8" explicitly ensures cross-platform compatibility.
| data = json.loads(data_path.read_text()) | |
| original = html_path.read_text() | |
| data = json.loads(data_path.read_text(encoding="utf-8")) | |
| original = html_path.read_text(encoding="utf-8") |
| return 0 | ||
|
|
||
| if new_text != original: | ||
| html_path.write_text(new_text) |
There was a problem hiding this comment.
Similarly, Path.write_text() should explicitly specify encoding="utf-8" to ensure the output file is written with UTF-8 encoding across all platforms.
| html_path.write_text(new_text) | |
| html_path.write_text(new_text, encoding="utf-8") |
| data = json.loads((ss.REPO_ROOT / "data/site.json").read_text()) | ||
| original = (ss.REPO_ROOT / "index.html").read_text() |
There was a problem hiding this comment.
Specify encoding="utf-8" here as well to prevent UnicodeDecodeError on Windows environments during test execution.
| data = json.loads((ss.REPO_ROOT / "data/site.json").read_text()) | |
| original = (ss.REPO_ROOT / "index.html").read_text() | |
| data = json.loads((ss.REPO_ROOT / "data/site.json").read_text(encoding="utf-8")) | |
| original = (ss.REPO_ROOT / "index.html").read_text(encoding="utf-8") |
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The
INDENT = " " * 12constant tightly couplesrender_organsto the current HTML formatting; consider deriving the indent from the matched<!-- organs:start -->line (e.g., via a capturing group on leading whitespace) to make the script resilient to future formatting changes. - The
VALUEmarker regex only allows[A-Za-z0-9_]in keys, which prevents using more descriptive marker names with hyphens or dots; you might want to broaden this character class if you expect more flexible key naming indata['values'].
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `INDENT = " " * 12` constant tightly couples `render_organs` to the current HTML formatting; consider deriving the indent from the matched `<!-- organs:start -->` line (e.g., via a capturing group on leading whitespace) to make the script resilient to future formatting changes.
- The `VALUE` marker regex only allows `[A-Za-z0-9_]` in keys, which prevents using more descriptive marker names with hyphens or dots; you might want to broaden this character class if you expect more flexible key naming in `data['values']`.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 006d117766
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if args.check: | ||
| if new_text != original: | ||
| print("error: index.html is out of sync with data/site.json — " | ||
| "run `python3 scripts/sync-site.py`", file=sys.stderr) | ||
| return 1 | ||
| print("index.html is in sync.") | ||
| return 0 |
There was a problem hiding this comment.
--check only compares new_text to original, so it returns success even when rendering emits warnings that the HTML no longer contains required markers. If index.html loses <!-- v:... --> or <!-- organs:start/end -->, no replacement occurs, new_text == original, and CI reports “in sync” while synchronization is actually broken. This creates a false negative that defeats the single-source-of-truth guardrail.
Useful? React with 👍 / 👎.
| - name: Run tests | ||
| run: python3 scripts/test_sync_site.py |
There was a problem hiding this comment.
In the push/manual path, the workflow executes python3 scripts/test_sync_site.py before the sync step, but that test suite includes test_real_index_in_sync, which fails whenever data/site.json changed without an updated index.html. Because GitHub Actions stops on step failure, the later “Sync and commit” step never runs in the exact drift scenario it is meant to auto-fix.
Useful? React with 👍 / 👎.
| paths: | ||
| - "data/site.json" | ||
| - "scripts/sync-site.py" | ||
| - ".github/workflows/site-sync.yml" |
There was a problem hiding this comment.
The push trigger only watches data/site.json, scripts/sync-site.py, and the workflow file, so a direct push that edits only index.html will not run this workflow at all. That allows generated content drift on main with no check or auto-repair, which undermines the intended single-source-of-truth enforcement.
Useful? React with 👍 / 👎.
|
|
||
| if new_text != original: | ||
| html_path.write_text(new_text) | ||
| print(f"updated {html_path.relative_to(REPO_ROOT)}") |
There was a problem hiding this comment.
The script documents SITE_HTML as an override, but when a changed file is outside REPO_ROOT, html_path.relative_to(REPO_ROOT) raises ValueError and exits non-zero after writing. This breaks valid override usage and makes automation treat a successful sync as a failure.
Useful? React with 👍 / 👎.
| if key not in values: | ||
| warnings.append(f"marker '{key}' has no entry in data['values']") | ||
| return m.group(0) | ||
| return f"{m.group(1)}{values[key]}{m.group(4)}" |
There was a problem hiding this comment.
The value replacement path writes values[key] directly into HTML comments without escaping, unlike the organ renderer which escapes both labels and URLs. If any data['values'] entry contains <, &, or quotes, the generated page can become malformed or inject unintended markup instead of plain text, which is a data-driven correctness and safety regression.
Useful? React with 👍 / 👎.
Read/write files as UTF-8 (index.html has em-dashes), and derive the nav block's indent from the matched <!-- organs:start --> marker instead of a hardcoded 12-space constant, so the script survives HTML reflows. Add tests for explicit indent and marker-derived indent. Addresses gemini-code-assist and sourcery-ai feedback on #3. https://claude.ai/code/session_017iav9RLCuHkepAqnGWYMKo
|
Addressed the review feedback in
On the Generated by Claude Code |
--check now fails on warnings (deleted marker / orphan value), closing a false-negative. HTML-escape value-marker content (parity with the organ renderer). Use a repo-relative display helper so a SITE_HTML override outside the repo root no longer crashes after writing. On the push/manual path, run the sync before the in-sync guard test so the drift it repairs can't kill the job; add index.html to the push triggers. Add a --check missing-marker test. Addresses chatgpt-codex-connector review feedback on #3. https://claude.ai/code/session_017iav9RLCuHkepAqnGWYMKo
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a60b66ee08
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| out, _ = ss.render(original, data) | ||
| self.assertEqual(out, original, "index.html is out of sync with data/site.json") |
There was a problem hiding this comment.
The test_real_index_in_sync guard only checks out == original and ignores warnings from render(). In the push/manual workflow path, this lets CI pass even if required markers were removed from index.html (for example <!-- v:synced --> or <!-- organs:start/end -->): sync-site.py warns but exits 0 in write mode, and this test still passes because no textual replacement occurs. That means the auto-repair job can report success while synchronization is actually broken on main.
Useful? React with 👍 / 👎.
| if key not in seen: | ||
| warnings.append(f"value '{key}' has no matching marker in the HTML") | ||
|
|
||
| if "organs" in data: |
There was a problem hiding this comment.
render() only validates the nav block when the organs key exists, so if data/site.json drops or misspells organs, the existing HTML nav is left untouched and no warning is emitted. In --check mode this becomes a false “in sync” result even though the single source of truth for organ links is missing, which undermines the drift detection this script is intended to enforce.
Useful? React with 👍 / 👎.
2 participants
Summary
Normalizes the landing page's hardcoded dynamic data (organ nav URLs + footer sync date) into a single source of truth — part of the "normalize dynamic data" pass.
data/site.json— single source for the organ-nav entries (the*.github.ioURLs that duplicate the organ→org map) and the footer "System Synchronization" date.scripts/sync-site.py— renders that data intoindex.html:<!-- organs:start -->...<!-- organs:end -->nav block (HTML-escaped, preserves theactivestate),<!-- v:synced -->...<!-- /v -->footer date.--checkmode fails on drift;SITE_DATA/SITE_HTMLenv overrides.scripts/test_sync_site.py—unittestcoverage for rendering, escaping, idempotency, orphan handling, and a guard asserting the committedindex.htmlalready matches the committed data..github/workflows/site-sync.yml— runs the tests, verifies sync on PRs, and auto-commits regenerated HTML on push/dispatch..gitignore— ignore Python bytecode.No visible change: the rendered
index.htmlis byte-identical tomainapart from the marker comments (the data file holds the current values).Test plan
python3 scripts/test_sync_site.py— 7 passingpython3 scripts/sync-site.py --check— in sync (exit 0)git diff main -- index.htmlshows only the added marker comments--checkexits 1; write mode fixes it; restore leaves the page unchangedsite-syncworkflow green on this PRhttps://claude.ai/code/session_017iav9RLCuHkepAqnGWYMKo
Generated by Claude Code
Summary by Sourcery
Normalize the landing page’s organ navigation and footer sync date to be generated from a shared data file and enforced via automation.
New Features:
Enhancements:
CI:
Tests:
Chores: