-
Notifications
You must be signed in to change notification settings - Fork 14
feat(edge-cve): add Black CVE investigation plugin with govulncheck #232
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
eggfoobar
wants to merge
4
commits into
openshift-eng:main
Choose a base branch
from
eggfoobar:add-cve-scanner
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
fc14bcf
feat(edge-cve): add Black CVE investigation plugin with govulncheck
eggfoobar d84bbf8
fix(edge-cve): address CodeRabbit review findings for scan safety and…
eggfoobar af2f38a
fix(edge-cve): harden scan/report paths from CodeRabbit follow-ups
eggfoobar c56ac4a
fix(edge-cve): silence shellcheck SC2329 and ruff S603 in tests
eggfoobar File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -12,3 +12,4 @@ markdownlint-cli2-results.json | |
| .env | ||
| plugins/edge-ocp-rc/jobs/*.txt | ||
| node_modules/ | ||
| .work | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| { | ||
| "name": "edge-cve", | ||
| "description": "Investigate open Black CVE Jira tickets with govulncheck scans and actionable remediation reports", | ||
| "version": "1.0.0", | ||
| "author": { | ||
| "name": "edge-tooling" | ||
| }, | ||
| "homepage": "https://github.com/openshift-eng/edge-tooling", | ||
| "license": "Apache-2.0" | ||
| } |
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| { | ||
| "defaults": { | ||
| "host": "github.com", | ||
| "org": "openshift" | ||
| }, | ||
| "components": { | ||
| "MicroShift": { | ||
| "repo": "openshift/microshift", | ||
| "language": "go", | ||
| "version_ref_template": "release-{version}", | ||
| "version_ref_fallbacks": [] | ||
| }, | ||
| "Logical Volume Manager Storage": { | ||
| "repo": "openshift/lvm-operator", | ||
| "language": "go", | ||
| "version_ref_template": "release-{version}", | ||
| "version_ref_fallbacks": [] | ||
| }, | ||
| "Two Node Fencing": { | ||
| "repo": "openshift-eng/two-node-toolbox", | ||
| "language": "go", | ||
| "version_ref_template": "main", | ||
| "version_ref_fallbacks": [] | ||
| }, | ||
| "Two Node Arbiter": { | ||
| "repo": "openshift-eng/two-node-toolbox", | ||
| "language": "go", | ||
| "version_ref_template": "main", | ||
| "version_ref_fallbacks": [] | ||
| }, | ||
| "Cluster Node Tuning Operator": { | ||
| "repo": "openshift/cluster-node-tuning-operator", | ||
| "language": "go", | ||
| "version_ref_template": "release-{version}", | ||
| "version_ref_fallbacks": [] | ||
| }, | ||
| "Machine Config Operator": { | ||
| "repo": "openshift/machine-config-operator", | ||
| "language": "go", | ||
| "version_ref_template": "release-{version}", | ||
| "version_ref_fallbacks": [] | ||
| } | ||
| }, | ||
| "repo_url_patterns": [ | ||
| "github\\.com/(?P<org>[^/\\s]+)/(?P<repo>[^/\\s#?]+)", | ||
| "git@github\\.com:(?P<org>[^/\\s]+)/(?P<repo>[^/\\s#?.]+)" | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| apiVersion: batch/v1 | ||
| kind: Job | ||
| metadata: | ||
| name: govulncheck-__TARGET_ID__ | ||
| namespace: edge-cve-scans | ||
| labels: | ||
| app.kubernetes.io/name: edge-cve-govulncheck | ||
| edge-cve/target-id: "__TARGET_ID__" | ||
| edge-cve/repo: "__REPO_LABEL__" | ||
| spec: | ||
| backoffLimit: 1 | ||
| ttlSecondsAfterFinished: 86400 | ||
| template: | ||
| metadata: | ||
| labels: | ||
| app.kubernetes.io/name: edge-cve-govulncheck | ||
| edge-cve/target-id: "__TARGET_ID__" | ||
| spec: | ||
| restartPolicy: Never | ||
| serviceAccountName: edge-cve-scanner | ||
| automountServiceAccountToken: true | ||
| securityContext: | ||
| runAsNonRoot: true | ||
| containers: | ||
| - name: govulncheck | ||
| image: registry.redhat.io/ubi9/go-toolset:1.23 | ||
| securityContext: | ||
| runAsNonRoot: true | ||
| allowPrivilegeEscalation: false | ||
| readOnlyRootFilesystem: true | ||
| capabilities: | ||
| drop: | ||
| - ALL | ||
| env: | ||
| - name: REPO_URL | ||
| value: "__REPO_URL__" | ||
| - name: REPO_SLUG | ||
| value: "__REPO_SLUG__" | ||
| - name: REPO_LABEL | ||
| value: "__REPO_LABEL__" | ||
| - name: GIT_REF | ||
| value: "__GIT_REF__" | ||
| - name: TARGET_ID | ||
| value: "__TARGET_ID__" | ||
| - name: CVE_IDS | ||
| value: "__CVE_IDS__" | ||
| - name: TICKET_KEYS | ||
| value: "__TICKET_KEYS__" | ||
| - name: HOME | ||
| value: "/tmp" | ||
| - name: GOPATH | ||
| value: "/tmp/go" | ||
| - name: GOCACHE | ||
| value: "/tmp/go/cache" | ||
| - name: GOMODCACHE | ||
| value: "/tmp/go/pkg/mod" | ||
| - name: GOTOOLCHAIN | ||
| value: "auto" | ||
| resources: | ||
| requests: | ||
| cpu: "1" | ||
| memory: "4Gi" | ||
| ephemeral-storage: "4Gi" | ||
| limits: | ||
| cpu: "3" | ||
| memory: "6Gi" | ||
| ephemeral-storage: "6Gi" | ||
| volumeMounts: | ||
| # Writable /tmp for HOME/GOPATH/GOCACHE/GOMODCACHE and | ||
| # govulncheck stdout/stderr files (required with readOnlyRootFilesystem). | ||
| - name: tmp | ||
| mountPath: /tmp | ||
| - name: workspace | ||
| mountPath: /tmp/workspace | ||
| - name: scripts | ||
| mountPath: /scripts | ||
| readOnly: true | ||
| command: | ||
| - /bin/bash | ||
| - /scripts/scan_target.sh | ||
| activeDeadlineSeconds: 1800 | ||
| volumes: | ||
| - name: tmp | ||
| emptyDir: | ||
| sizeLimit: 3Gi | ||
| - name: workspace | ||
| emptyDir: | ||
| sizeLimit: 3Gi | ||
| - name: scripts | ||
| configMap: | ||
| name: edge-cve-govulncheck-scripts | ||
| defaultMode: 0444 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| apiVersion: v1 | ||
| kind: Namespace | ||
| metadata: | ||
| name: edge-cve-scans | ||
| labels: | ||
| app.kubernetes.io/name: edge-cve-scans |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| apiVersion: v1 | ||
| kind: ServiceAccount | ||
| metadata: | ||
| name: edge-cve-scanner | ||
| namespace: edge-cve-scans | ||
| --- | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: Role | ||
| metadata: | ||
| name: edge-cve-scanner | ||
| namespace: edge-cve-scans | ||
| rules: | ||
| - apiGroups: [""] | ||
| resources: ["configmaps"] | ||
| verbs: ["get", "list", "create", "patch", "update"] | ||
| --- | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: RoleBinding | ||
| metadata: | ||
| name: edge-cve-scanner | ||
| namespace: edge-cve-scans | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: edge-cve-scanner | ||
| namespace: edge-cve-scans | ||
| roleRef: | ||
| kind: Role | ||
| name: edge-cve-scanner | ||
| apiGroup: rbac.authorization.k8s.io |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| requests>=2.28.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,183 @@ | ||
| #!/usr/bin/env python3 | ||
| """Deterministically analyze a single govulncheck result.json and decide what | ||
| action, if any, is needed - no LLM call required for this base determination. | ||
|
|
||
| Given the result.json written by process_govulncheck_result.go (local mode; | ||
| see run_single_repo_scan.sh / scan_target.sh), this: | ||
|
|
||
| 1. Computes a verdict ("affected", "not_affected", or "inconclusive") using | ||
| the same signal-kill-aware logic as generate_report.py's | ||
| verdict_for_ticket, so a scan that was OOM-killed (scan_incomplete) is | ||
| never mistaken for a clean "not affected" result. | ||
| 2. Builds a ready-to-use `suggested_agent_prompt` string from a fixed | ||
| template filled in with the scan's own matched findings - a deterministic | ||
| remediation prompt, not an LLM-generated one. Callers who want the LLM to | ||
| refine/verify this prompt (e.g. edge-cve:investigate Step 3) can still do | ||
| so as a separate step. | ||
|
|
||
| Usage: | ||
| analyze_scan_result.py --result RESULT_JSON [--out OUT_JSON] | ||
| [--jira-url URL] [--summary TEXT] [--component NAME] | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import json | ||
| import sys | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| def determine_verdict(result: dict) -> tuple[str, bool]: | ||
| """Return (verdict, action_required).""" | ||
| if result.get("scan_incomplete"): | ||
| return "inconclusive", False | ||
| if result.get("affected"): | ||
| return "affected", True | ||
| # govulncheck: 0 = clean, 3 = vulnerabilities found. Any other exit is | ||
| # abnormal (tool/build error, etc.) - treat as inconclusive even when | ||
| # finding_count is 0, so a failed scan is never mistaken for "not affected". | ||
| if result.get("scan_exit_code", 0) not in (0, 3): | ||
| return "inconclusive", False | ||
| return "not_affected", False | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
|
|
||
| def finding_label(finding: dict) -> str: | ||
| inner = finding.get("finding", finding) | ||
| if not isinstance(inner, dict): | ||
| inner = finding if isinstance(finding, dict) else {} | ||
| # govulncheck may emit finding.osv as a string ID or an embedded OSV object. | ||
| osv = inner.get("osv") | ||
| if not osv: | ||
| osv = inner.get("vulnerability") | ||
| vuln_id = "?" | ||
| if isinstance(osv, str): | ||
| vuln_id = osv or "?" | ||
| elif isinstance(osv, dict): | ||
| raw_id = osv.get("id", "?") | ||
| vuln_id = raw_id if isinstance(raw_id, str) and raw_id else "?" | ||
| module = "" | ||
| trace = inner.get("trace") or [] | ||
| if trace and isinstance(trace, list) and isinstance(trace[0], dict): | ||
| module = trace[0].get("module", "") or trace[0].get("package", "") or "" | ||
| return f"{vuln_id}" + (f" in {module}" if module else "") | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
|
|
||
| def build_prompt( | ||
| result: dict, | ||
| verdict: str, | ||
| *, | ||
| jira_url: str = "", | ||
| summary: str = "", | ||
| component: str = "", | ||
| ) -> str | None: | ||
| if verdict == "not_affected": | ||
| return None | ||
|
|
||
| repo_url = result.get("repo_url", "") | ||
| repo_slug = result.get("repo_slug", "") | ||
| git_ref = result.get("git_ref", "") | ||
| commit = result.get("commit", "") or "" | ||
| cve_ids = result.get("cve_ids") or [] | ||
| findings = result.get("matched_findings") or [] | ||
|
|
||
| lines = [ | ||
| "You are fixing a CVE in an OpenShift edge component repository.", | ||
| "", | ||
| ] | ||
| if jira_url: | ||
| lines.append(f"Jira: {jira_url}") | ||
| if summary: | ||
| lines.append(f"Summary: {summary}") | ||
| if cve_ids: | ||
| lines.append(f"CVEs: {', '.join(cve_ids)}") | ||
| if component: | ||
| lines.append(f"Component: {component}") | ||
| lines.append(f"Repository: {repo_slug} ({repo_url})") | ||
| lines.append(f"Target ref: {git_ref} (commit {commit[:12] if commit else 'unknown'})") | ||
| lines.append("") | ||
|
|
||
| if verdict == "inconclusive": | ||
| lines.extend( | ||
| [ | ||
| "govulncheck did not produce a conclusive result for this ref", | ||
| "(scan_incomplete or a non-zero exit with ambiguous findings), so this", | ||
| "is NOT yet confirmed as affected. Before writing any fix:", | ||
| "1. Re-run with more memory/CPU (see run_single_repo_scan.sh/" | ||
| "run_govulncheck_podman.sh --memory) or check the scan's stderr_tail" | ||
| " for the real cause.", | ||
| "2. Only proceed with a fix once govulncheck confirms an affected finding.", | ||
| ] | ||
| ) | ||
| return "\n".join(lines) | ||
|
|
||
| # verdict == "affected" | ||
| lines.append("govulncheck confirmed this repository/ref is affected:") | ||
| for finding in findings: | ||
| lines.append(f"- {finding_label(finding)}") | ||
| lines.extend( | ||
| [ | ||
| "", | ||
| "Steps:", | ||
| f"1. Clone the repository and checkout {git_ref}.", | ||
| "2. Confirm the vulnerable module/path above against govulncheck's " | ||
| "call-graph findings (matched_findings in the scan result).", | ||
| "3. Bump the dependency (or apply the upstream fix) to a version " | ||
| "that resolves the vulnerability.", | ||
| "4. Run `go mod tidy && go test ./...` and `govulncheck ./...` to " | ||
| "verify the fix and check for regressions.", | ||
| "5. Open a PR" | ||
| + (f" referencing {jira_url}" if jira_url else " describing the fix") | ||
| + ".", | ||
| ] | ||
| ) | ||
| return "\n".join(lines) | ||
|
|
||
|
|
||
| def main() -> None: | ||
| parser = argparse.ArgumentParser(description=__doc__) | ||
| parser.add_argument("--result", required=True, help="Path to a result.json from a govulncheck scan") | ||
| parser.add_argument("--out", help="Write the augmented JSON here (always also printed to stdout)") | ||
| parser.add_argument("--jira-url", default="", help="Jira ticket URL for context in the prompt") | ||
| parser.add_argument("--summary", default="", help="Ticket/issue summary for context in the prompt") | ||
| parser.add_argument("--component", default="", help="Component name for context in the prompt") | ||
| args = parser.parse_args() | ||
|
|
||
| result_path = Path(args.result) | ||
| if not result_path.is_file(): | ||
| print(f"Error: {result_path} not found", file=sys.stderr) | ||
| sys.exit(1) | ||
|
|
||
| result = json.loads(result_path.read_text(encoding="utf-8")) | ||
| verdict, action_required = determine_verdict(result) | ||
| prompt = build_prompt( | ||
| result, | ||
| verdict, | ||
| jira_url=args.jira_url, | ||
| summary=args.summary, | ||
| component=args.component, | ||
| ) | ||
|
|
||
| output = dict(result) | ||
| output["verdict"] = verdict | ||
| output["action_required"] = action_required | ||
| output["suggested_agent_prompt"] = prompt | ||
| # Persist the context args as their own fields (not just baked into the | ||
| # prompt text) so downstream consumers (generate_html_report.py) can | ||
| # render a proper Jira link/summary without re-parsing prose. | ||
| if args.jira_url: | ||
| output["jira_url"] = args.jira_url | ||
| if args.summary: | ||
| output["summary"] = args.summary | ||
| if args.component: | ||
| output["component"] = args.component | ||
|
|
||
| text = json.dumps(output, indent=2) | ||
| if args.out: | ||
| Path(args.out).write_text(text + "\n", encoding="utf-8") | ||
| print(f"Written: {args.out}", file=sys.stderr) | ||
| print(text) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.