Skip to content

fix: address critical and high CVE findings - #1103

Open
mckornfield wants to merge 3 commits into
mainfrom
address-cve-findings/rsadler
Open

fix: address critical and high CVE findings#1103
mckornfield wants to merge 3 commits into
mainfrom
address-cve-findings/rsadler

Conversation

@mckornfield

@mckornfield mckornfield commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

  • address Pulse OSS findings from Platform-Deploy security.yml run 31002650434
  • update Python, Go, and Studio web dependencies for reported Critical/High rows
  • extend container cleanup for Perl/Pillow findings

CVE Notes

  • No additional pre-PR CVE refresh is needed for this remediation PR; the branch is based on the available Pulse container and OSS artifacts.
  • A fresh container build and Pulse scan is still required after CI/build artifacts exist to prove the container-only rows disappear.
  • PR fix(customizer): fix CVEs + vllm venv issue #1056 remains relevant RL context, but is not merged into this branch.
  • Remaining OSS LangChain/ragas rows are not addressed here because langchain-community<0.4 is currently capped for ragas 0.4.3 compatibility, and the Pulse OSS report gives ragas unknown upgrade guidance.

Verification

  • uv lock --check
  • cd services/core/jobs/jobs-launcher && go test ./...
  • cd services/guardrails/callouts && go test ./...
  • cd web && pnpm install --lockfile-only
  • rg -n "vitest@4\\.1\\.9|@vitest/(coverage-v8|ui)@4\\.1\\.9|vitest: \\^4\\.1\\.9|version: 4\\.1\\.9" web/pnpm-workspace.yaml web/pnpm-lock.yaml
  • git diff --check origin/main...HEAD
  • bash -n docker/scripts/cve-cleanup.sh
  • docker buildx bake --print nmp-cpu-tasks-docker

Caveats

  • Studio tests were not run locally because this machine has Node v22.18.0, below the repo requirement >=22.23.2 <23.
  • web/node_modules was not cleaned or refreshed; only the lockfile was updated.

Summary by CodeRabbit

  • Bug Fixes

    • Addressed security-related vulnerabilities in runtime images by removing stale and affected package files.
    • Improved container build cleanup for more consistent runtime environments.
  • Maintenance

    • Updated security and development tooling to newer supported versions.
    • Refreshed runtime dependency requirements across task, training, and automation images.
    • Updated gRPC and frontend test tooling for improved compatibility and reliability.

Signed-off-by: Matt Kornfield <mkornfield@nvidia.com>
@mckornfield
mckornfield requested review from a team as code owners August 5, 2026 19:45
@github-actions github-actions Bot added the fix label Aug 5, 2026
Signed-off-by: Matt Kornfield <mkornfield@nvidia.com>
@github-actions

github-actions Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor
Suite Lines Covered Line Rate Branch Rate
Unit Tests 30743/39237 78.3% 62.7%
Integration Tests 18062/37189 48.6% 21.0%

@mckornfield
mckornfield requested a review from ironcommit August 5, 2026 19:57
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 715b3fff-572d-4aa8-bd39-fc2bbacc0e46

📥 Commits

Reviewing files that changed from the base of the PR and between e55d517 and fd6f6df.

📒 Files selected for processing (1)
  • docker/automodel/Dockerfile.nmp-automodel-base
🚧 Files skipped from review as they are similar to previous changes (1)
  • docker/automodel/Dockerfile.nmp-automodel-base

📝 Walkthrough

Walkthrough

The changes raise dependency minimums and update Docker image remediation. Runtime builds now install the CVE cleanup script, pin Pillow, and remove stale Pillow files.

Changes

Dependency and image remediation

Layer / File(s) Summary
Dependency constraint updates
pyproject.toml, docker/Dockerfile.auditor-tasks, docker/automodel/Dockerfile.nmp-automodel-base, services/core/models/pyproject.toml, services/guardrails/callouts/go.mod, web/pnpm-workspace.yaml
Dependency minimums and catalog versions were raised for GitPython, pyasn1, datamodel-code-generator, gRPC, and Vitest packages.
Docker image remediation
docker/Dockerfile.nmp-cpu-tasks, docker/Dockerfile.nmp-unsloth-training, docker/automodel/Dockerfile.nmp-automodel-base, docker/scripts/cve-cleanup.sh
Docker builds now run the CVE cleanup script, install constrained Pillow versions, and remove stale Pillow packages, native libraries, and metadata.

Possibly related PRs

Suggested reviewers: ironcommit, crookedstorm

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the pull request's primary purpose of addressing critical and high CVE findings.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch address-cve-findings/rsadler

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
CVE_SUMMARY.md (1)

1-5: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add the required documentation structure.

This page records facts and verification evidence, so classify it as REFERENCE. Put a short status summary and a ## Prerequisites section before ## Artifacts Reviewed. Add a ## Next Steps section after ## Verification for the required image rescan and Studio tests.

As per coding guidelines, Markdown pages must fit one Diataxis quadrant, list prerequisites at the top, and include a Next Steps section at the end.

Also applies to: 96-97

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CVE_SUMMARY.md` around lines 1 - 5, Add the required REFERENCE-page structure
in CVE_SUMMARY.md: place a concise status summary and a ## Prerequisites section
before ## Artifacts Reviewed, and add ## Next Steps after ## Verification
describing the required image rescan and Studio tests. Keep the page focused on
factual evidence and verification.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@CVE_SUMMARY.md`:
- Around line 77-78: Update the verification command in CVE_SUMMARY.md to invoke
summarize_findings.py through a repository-relative path or another portable
tool location, removing the host-specific /home/mkornfield/home/skills prefix.
Verify the documented command works from a fresh checkout before retaining it.
- Around line 49-51: Update the `nmp-cpu-tasks`, `nmp-automodel-training`, and
`nmp-unsloth-training` entries in `CVE_SUMMARY.md` to describe their
remediations as expected or pending rather than confirmed. Reserve definitive
“addressed” wording until rebuilt images receive a fresh Pulse scan confirming
the package fixes and stale-file removal.

In `@docker/scripts/cve-cleanup.sh`:
- Around line 90-92: Update the Pillow cleanup command in the CVE cleanup script
to remove the packages from the active /app/.venv Python 3.13 environment
instead of the Python 3.12 system paths, covering the Pillow package, native
libraries, and dist-info metadata; then verify the rebuilt CPU image no longer
contains the vulnerable installation.

In `@pyproject.toml`:
- Line 269: Update the Automodel image dependency constraints in its Dockerfile
to require GitPython>=3.1.57,<4 and pyasn1>=0.6.4,<1, matching the project
dependency floors. Rebuild the image and run the image security scan to verify
the updated constraints.

---

Nitpick comments:
In `@CVE_SUMMARY.md`:
- Around line 1-5: Add the required REFERENCE-page structure in CVE_SUMMARY.md:
place a concise status summary and a ## Prerequisites section before ##
Artifacts Reviewed, and add ## Next Steps after ## Verification describing the
required image rescan and Studio tests. Keep the page focused on factual
evidence and verification.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: ed093274-8705-4a3c-b777-6cc53d020f37

📥 Commits

Reviewing files that changed from the base of the PR and between 77f890f and e55d517.

⛔ Files ignored due to path filters (3)
  • services/guardrails/callouts/go.sum is excluded by !**/*.sum
  • uv.lock is excluded by !**/*.lock
  • web/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (10)
  • CVE_SUMMARY.md
  • docker/Dockerfile.auditor-tasks
  • docker/Dockerfile.nmp-cpu-tasks
  • docker/Dockerfile.nmp-unsloth-training
  • docker/automodel/Dockerfile.nmp-automodel-base
  • docker/scripts/cve-cleanup.sh
  • pyproject.toml
  • services/core/models/pyproject.toml
  • services/guardrails/callouts/go.mod
  • web/pnpm-workspace.yaml

Comment thread CVE_SUMMARY.md Outdated
Comment on lines +49 to +51
- `nmp-cpu-tasks` Perl package rows: 12 Critical and 16 High rows across `perl-base`, `perl`, `perl-modules-5.36`, and `libperl5.36` are addressed by the runtime cleanup step.
- `nmp-automodel-training` Pillow rows: 10 High rows for stale `/usr/local/lib/python3.12/dist-packages/pillow-12.2.0` copies are addressed by removing shadowed system-site Pillow.
- `nmp-unsloth-training` Pillow rows: 10 High rows for stale `/usr/local/lib/python3.12/dist-packages/pillow-12.2.0` copies are addressed by installing Pillow `>=12.3.0,<13` into `/opt/venv` and removing the shadowed system copy.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/sh
set -eu

file=$(git ls-files | awk '$0 == "CVE_SUMMARY.md" {print; exit}')
test -n "$file"

printf '%s\n' '--- CVE_SUMMARY.md (relevant sections) ---'
sed -n '1,115p' "$file"

printf '%s\n' '--- references to rebuild, rescan, addressed, and Pulse ---'
rg -n -i 'rebuild|rescan|addressed|expected|Pulse|scan' "$file"

Repository: NVIDIA-NeMo/nemo-platform

Length of output: 163


🏁 Script executed:

#!/bin/sh
set -u

printf '%s\n' '--- repository root ---'
pwd

printf '%s\n' '--- tracked matching files ---'
git ls-files | grep -E '(^|/)CVE_SUMMARY\.md$' || true

printf '%s\n' '--- candidate file locations ---'
find . -type f -name 'CVE_SUMMARY.md' -print

printf '%s\n' '--- candidate file content ---'
for file in $(find . -type f -name 'CVE_SUMMARY.md' -print); do
  sed -n '1,115p' "$file"
  printf '%s\n' '--- references ---'
  grep -nEi 'rebuild|rescan|addressed|expected|Pulse|scan' "$file" || true
done

Repository: NVIDIA-NeMo/nemo-platform

Length of output: 301


Security Misconfiguration (CWE-693)

Reachability: Internal · Exploitability: Moderate

Separate intended remediation from scan-confirmed remediation.

Because Lines 49–51 state that the findings are addressed while Line 96 states that the images were not rebuilt or rescanned, change these entries to “expected to be addressed” or move them to a pending section. Close them only after a fresh Pulse scan confirms the fixed packages and removal of stale files.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@CVE_SUMMARY.md` around lines 49 - 51, Update the `nmp-cpu-tasks`,
`nmp-automodel-training`, and `nmp-unsloth-training` entries in `CVE_SUMMARY.md`
to describe their remediations as expected or pending rather than confirmed.
Reserve definitive “addressed” wording until rebuilt images receive a fresh
Pulse scan confirming the package fixes and stale-file removal.

Comment thread CVE_SUMMARY.md Outdated
Comment on lines +90 to +92
rm -rf /usr/local/lib/python3.12/dist-packages/PIL \
/usr/local/lib/python3.12/dist-packages/pillow.libs \
/usr/local/lib/python3.12/dist-packages/pillow-*.dist-info

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

rg -n -C 2 'nmp-python-base|python:3\.[0-9]+' \
  docker/Dockerfile.nmp-cpu-tasks docker CVE_SUMMARY.md
rg -n 'dist-packages|PIL|pillow' \
  docker/scripts/cve-cleanup.sh docker --glob 'Dockerfile*'

Repository: NVIDIA-NeMo/nemo-platform

Length of output: 3995


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- base image and shared script ---'
cat -n docker/base/Dockerfile.nmp-python-base | sed -n '1,90p'
cat -n docker/scripts/cve-cleanup.sh | sed -n '1,130p'

printf '%s\n' '--- cleanup callers and Pillow installation paths ---'
rg -n -C 3 'cve-cleanup\.sh|Pillow|pillow|dist-packages|site-packages|uv sync|pip install' docker

Repository: NVIDIA-NeMo/nemo-platform

Length of output: 50381


🏁 Script executed:

#!/bin/bash
set -euo pipefail

python3 - <<'PY'
from pathlib import Path
import re

base = Path("docker/base/Dockerfile.nmp-python-base").read_text()
cpu = Path("docker/Dockerfile.nmp-cpu-tasks").read_text()
cleanup = Path("docker/scripts/cve-cleanup.sh").read_text()

assert "python:3.13.14-slim-trixie" in base
assert "uv venv --seed --python=3.13 /app/.venv" in base
assert "uv sync --frozen --only-group cpu-tasks --no-editable" in cpu
assert "COPY docker/scripts/cve-cleanup.sh /bin/" in cpu
assert "RUN bash /bin/cve-cleanup.sh" in cpu

cleanup_paths = re.findall(r"/usr/local/lib/python[\d.]+/dist-packages/[^\s\\]+", cleanup)
print("base_python=3.13")
print("cpu_venv=/app/.venv")
print("expected_cpu_pillow_paths:")
for name in ("PIL", "pillow.libs", "pillow-*.dist-info"):
    print(f"  /app/.venv/lib/python3.13/site-packages/{name}")
print("cleanup_pillow_paths:")
for path in cleanup_paths:
    if any(part in path.lower() for part in ("pil", "pillow")):
        print(f"  {path}")
print("cpu_pillow_paths_covered_by_cleanup=False")
PY

cat -n docker/Dockerfile.nmp-cpu-tasks | sed -n '1,30p'

Repository: NVIDIA-NeMo/nemo-platform

Length of output: 1860


Security Misconfiguration (CWE-693)

Reachability: Internal · Exploitability: Theoretical

Use the active virtual environment for Pillow cleanup.

The CPU image installs Python 3.13 packages in /app/.venv, but the script deletes only Python 3.12 system packages. Remove Pillow from the active environment and verify the rebuilt image.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docker/scripts/cve-cleanup.sh` around lines 90 - 92, Update the Pillow
cleanup command in the CVE cleanup script to remove the packages from the active
/app/.venv Python 3.13 environment instead of the Python 3.12 system paths,
covering the Pillow package, native libraries, and dist-info metadata; then
verify the rebuilt CPU image no longer contains the vulnerable installation.

Comment thread pyproject.toml
Signed-off-by: Matt Kornfield <mkornfield@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant