Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/validate-framework-registry.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Validate framework registry

on:
pull_request:
push:
branches:
- main
schedule:
- cron: "17 8 * * 1"
workflow_dispatch:

jobs:
validate-framework-registry:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Validate framework registry shape
run: ruby scripts/validate_framework_registry.rb

- name: Report stale framework references
run: ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ If your contribution adds or changes framework references, update

```bash
ruby scripts/validate_framework_registry.rb
ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
```

If your contribution changes CI/CD examples, update
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@ Validate framework provenance, versions, owners, and review dates with:

```bash
ruby scripts/validate_framework_registry.rb
ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
```

CI/CD examples for GitHub Actions, GitLab CI, Azure DevOps, Jenkins,
Expand Down
13 changes: 11 additions & 2 deletions docs/framework-reference-registry.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,14 @@ Validate the registry locally with:
ruby scripts/validate_framework_registry.rb
```

Future stale-framework checks should use `date_reviewed` and `owner` from this
registry instead of scraping individual skill files.
Report references whose `date_reviewed` value is older than the review policy
window with:

```bash
ruby scripts/validate_framework_registry.rb --stale --max-age-days 365
```

The scheduled `Validate framework registry` workflow runs this stale-reference
report weekly. Owners should refresh stale entries by confirming the source URL,
updating `version` when the upstream framework has changed, and setting a new
`date_reviewed`.
57 changes: 56 additions & 1 deletion scripts/validate_framework_registry.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@
# frozen_string_literal: true

require "yaml"
require "date"

ROOT = File.expand_path("..", __dir__)
REGISTRY_PATH = File.join(ROOT, "data", "frameworks.yaml")
REQUIRED_TOP_LEVEL = %w[schema_version last_reviewed required_families references].freeze
REQUIRED_ENTRY_FIELDS = %w[id family name version url date_reviewed owner aliases].freeze
REQUIRED_FAMILIES = %w[OWASP NIST MITRE CIS CVSS SSVC EPSS SLSA CycloneDX SPDX].freeze
DATE_PATTERN = /\A\d{4}-\d{2}-\d{2}\z/
DEFAULT_MAX_AGE_DAYS = 365

def rel(path)
path.delete_prefix("#{ROOT}#{File::SEPARATOR}")
Expand All @@ -28,6 +30,56 @@ def validate_string(value, label, errors)
errors << "#{label} must be a non-empty string" unless value.is_a?(String) && !value.empty?
end

def usage
warn "Usage: ruby scripts/validate_framework_registry.rb [--stale] [--max-age-days DAYS] [--as-of YYYY-MM-DD]"
exit 2
end

def parse_date_argument(value)
Date.iso8601(value)
rescue Date::Error
usage
end

def parse_args(argv)
options = {
stale: false,
max_age_days: DEFAULT_MAX_AGE_DAYS,
as_of: Date.today
}

until argv.empty?
case argv.shift
when "--stale"
options[:stale] = true
when "--max-age-days"
value = argv.shift
usage unless value&.match?(/\A\d+\z/)
options[:max_age_days] = value.to_i
when "--as-of"
value = argv.shift
usage unless value
options[:as_of] = parse_date_argument(value)
else
usage
end
end

options
end

def validate_staleness(entry, prefix, options, errors)
return unless options[:stale] && entry["date_reviewed"].to_s.match?(DATE_PATTERN)

reviewed = Date.iso8601(entry["date_reviewed"])
age_days = (options[:as_of] - reviewed).to_i
return if age_days <= options[:max_age_days]

errors << "#{prefix}: #{entry['id']} reviewed #{age_days} days ago; owner #{entry['owner']} must refresh by #{options[:max_age_days]} days"
end

options = parse_args(ARGV.dup)

errors = []
registry = load_registry(errors)

Expand Down Expand Up @@ -81,6 +133,8 @@ def validate_string(value, label, errors)
errors << "#{prefix}.date_reviewed must use YYYY-MM-DD"
end

validate_staleness(entry, prefix, options, errors)

aliases = entry["aliases"]
unless aliases.is_a?(Array) && !aliases.empty?
errors << "#{prefix}.aliases must be a non-empty array"
Expand All @@ -105,7 +159,8 @@ def validate_string(value, label, errors)
end

if errors.empty?
puts "OK: validated framework registry with #{references.size} reference(s)."
stale_suffix = options[:stale] ? " and no references older than #{options[:max_age_days]} days as of #{options[:as_of]}" : ""
puts "OK: validated framework registry with #{references.size} reference(s)#{stale_suffix}."
else
puts "FAIL: framework registry validation failed."
errors.each { |error| puts " - #{error}" }
Expand Down
Loading