Skip to content

[DBA-295] Migrate validate-agent-guidance from Bash to Python#79

Open
catstrike wants to merge 8 commits intomainfrom
ls/validate_agent_guidance_py
Open

[DBA-295] Migrate validate-agent-guidance from Bash to Python#79
catstrike wants to merge 8 commits intomainfrom
ls/validate_agent_guidance_py

Conversation

@catstrike
Copy link
Copy Markdown
Collaborator

Summary

Replaces the Bash validate-agent-guidance script with a typed Python module
that is easier to extend and maintain. Adds MdFile and ClaudeDir
dataclasses for structured, lazy-parsed access to skills and agents.

Changes

Python rewrite of the validator

Replaced scripts/validate-agent-guidance.sh with scripts/validate_agent_guidance.py.
Introduced MdFile (lazy-parsed markdown with cached properties) and
ClaudeDir (.claude/ directory with skill_dirs, skill_files,
agent_files listings). Adds YAML frontmatter validation: name and
description fields required, name must match the skill directory name.

Files
  • scripts/validate_agent_guidance.py
  • scripts/validate-agent-guidance.sh (deleted)

Tooling update

Updated make lint-skills and the pre-commit hook to invoke the Python script.

Files
  • Makefile
  • .pre-commit-config.yaml

Test Plan

  • make lint-skills passes
  • make check passes (ruff + mypy + pre-commit hook)

🤖 Generated with Claude Code

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the agent guidance/skills Tier-1 validator from a Bash script to a typed Python module, and updates local tooling (Makefile + pre-commit) to invoke the new validator via uv.

Changes:

  • Added scripts/validate_agent_guidance.py implementing skill structure, cross-reference, and eval JSON validation (plus YAML frontmatter checks).
  • Removed scripts/validate-agent-guidance.sh.
  • Updated make lint-skills and the validate-agent-guidance pre-commit hook to run the Python validator.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
scripts/validate_agent_guidance.py New Python implementation of the guidance/skills static validator (structure, references, eval JSON, frontmatter).
scripts/validate-agent-guidance.sh Deleted legacy Bash validator.
Makefile Points lint-skills at the new Python validator.
.pre-commit-config.yaml Updates the local hook to run the new Python validator via uv.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Copy Markdown
Member

@SimonKaran13 SimonKaran13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As copilot suggested, scripts/smoke-test-skills.sh still references validate-agent-guidance.sh. Let's make sure it's not referenced anywhere before merging

Copilot AI review requested due to automatic review settings March 27, 2026 10:15
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@catstrike catstrike force-pushed the ls/validate_agent_guidance_py branch from 7815c57 to a812709 Compare March 27, 2026 11:06
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 27, 2026 11:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings March 27, 2026 11:29
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 27, 2026 11:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

m = re.match(r"^---\r?\n(.*?)\r?\n---(?:\r?\n|$)", self._content, re.DOTALL)
if not m:
return {}
parsed = yaml.safe_load(m.group(1))
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MdFile.frontmatter calls yaml.safe_load(...) without handling yaml.YAMLError. Invalid frontmatter will crash the validator with a traceback instead of producing a clean validation error; catch YAML parse errors and surface them as a normal validation message.

Suggested change
parsed = yaml.safe_load(m.group(1))
try:
parsed = yaml.safe_load(m.group(1))
except yaml.YAMLError:
# Invalid YAML frontmatter: treat as missing frontmatter to avoid crashing.
return {}

Copilot uses AI. Check for mistakes.
Comment on lines +114 to +115
if fm.get("name") and fm["name"] != skill_name:
errors.append(f"{skill_name}/SKILL.md frontmatter 'name' ({fm['name']!r}) does not match directory name")
Copy link

Copilot AI Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate_skill_structure() only checks that name/description keys exist in frontmatter, not that they have non-empty values. With YAML, description: parses as None (and currently becomes 'None'), so empty required fields can slip through; validate the values are non-empty.

Suggested change
if fm.get("name") and fm["name"] != skill_name:
errors.append(f"{skill_name}/SKILL.md frontmatter 'name' ({fm['name']!r}) does not match directory name")
continue
value = fm.get(field)
# Require non-empty string values for required fields
if not isinstance(value, str) or not value.strip():
errors.append(
f"{skill_name}/SKILL.md frontmatter field '{field}' must be a non-empty string"
)
name_value = fm.get("name")
if isinstance(name_value, str):
name_stripped = name_value.strip()
if name_stripped and name_stripped != skill_name:
errors.append(
f"{skill_name}/SKILL.md frontmatter 'name' ({name_value!r}) does not match directory name"
)

Copilot uses AI. Check for mistakes.
Andrei Gasparian added 2 commits March 27, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants