Skip to content

Add documentation consistency tests#189

Open
Areen-09 wants to merge 2 commits into
ARPAHLS:mainfrom
Areen-09:feat/issue-183-doc-drift-guards
Open

Add documentation consistency tests#189
Areen-09 wants to merge 2 commits into
ARPAHLS:mainfrom
Areen-09:feat/issue-183-doc-drift-guards

Conversation

@Areen-09

Copy link
Copy Markdown

Description

This PR adds automated documentation drift guard tests to ensure repository documentation stays synchronized with the registered skills.

Changes

  • Added tests/test_registry_docs.py with checks for:
    • Documentation page exists for every registered skill.
    • Skills index stays in sync with the registry.
    • Example script index parity.
    • Agent loops documentation coverage.
  • Updated docs/TESTING.md to include tests/test_registry_docs.py in framework tests.

Type of Change (Matches Issue Templates)

  • Skill Proposal: New Skill (Contains manifest.yaml, skill.py, and instructions.md)
  • Bug Report Fix: Non-breaking change which fixes an execution error or framework bug
  • Doc Fix: Documentation Update
  • Framework Feature / RFC Updates: Core Framework Update (Changes to base_skill.py, loader.py, etc.)

Checklist (all PRs)

  • My code follows the Agent Code of Conduct.
  • I have run python -m flake8 ., pytest skills/ (or skillware test), and pytest tests/ locally (or the subset relevant to this change).
  • CHANGELOG.md updated under [Unreleased] if this PR changes user-visible behavior.
  • examples/README.md is updated if this PR adds, renames, or removes a runnable script under examples/.

New or updated skill (complete only if this PR adds or changes a skill under skills/)

Not applicable. This PR does not add or modify any skills.

Constitution & Safety (if adding or modifying a skill)

Not applicable. This PR does not add or modify any skills.

Related Issues

Closes #183

@rosspeili

Copy link
Copy Markdown
Contributor

Thanks @Areen-09, this is a solid start on #183. The four hard checks in tests/test_registry_docs.py match what we wanted, and they all pass on current main. Nice work on clear failure messages and the examples-index parity logic.

A few items before merge:

  1. CHANGELOG.md, please add an [Unreleased] entry under Tests or Added (user-visible CI guard).
  2. docs/usage/agent_loops.md, defi/evm_tx_handler is still missing from the reference matrix (called out in [Feat]: CI doc-drift guards — skill catalog and examples index parity #183). Please add a row (Gemini/Claude examples exist) so the soft check can pass, or document an explicit grandfather/TODO if you intentionally defer it. @Hendobox maybe you can guide @Areen-09 or co-author this pr?
  3. docs/TESTING.md, you updated the table row; also add a bullet under § Framework test mentioning tests/test_registry_docs.py (same style as test_skill_issuer.py).
  4. PR checklist, check Framework Feature, local flake8 / pytest tests/, and CHANGELOG when updated.
  5. Nit: typo shoudshould in the GRANDFATHERED_EXAMPLES comment.

Commits look good overall, once those are in, this should be good to merge. Happy to re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feat]: CI doc-drift guards — skill catalog and examples index parity

2 participants