fix(security): patch cryptography in structured lock; accept unfixable transformers advisories by RonShakutai · Pull Request #2158 · data-privacy-stack/presidio

RonShakutai · 2026-07-12T20:26:48Z

Change Description

Resolves the security findings that CI (Dependency Review + Trivy code scanning) reports on the committed uv.lock files introduced by #2152. Stacked on ronshakutai/fix-ci-poetry-timeout so #2152 can go green.

1. cryptography (real fix)

presidio-structured's lock resolved cryptography 46.0.7, which bundles a vulnerable OpenSSL (GHSA-537c-gmf6-5ccf, high). Added a uv constraint-dependencies pin (cryptography>=48.0.1); the lock now uses 49.0.0.

Side effect: the published presidio-anonymizer 2.2.363 caps cryptography <47, so structured's lock falls back to presidio-anonymizer 2.2.362 (which allows the patched line). This is a harmless CI-lock change — the repo's own anonymizer already declares cryptography>=48.0.1 (from fix(anonymizer): bump cryptography to >=48.0.1,<49.0.0 to resolve GHSA-537c-gmf6-5ccf #2144); a future anonymizer release lifts the cap.
Structured's test suite passes (28 passed).
Clears the Dependency Review failure and 1 of 3 Trivy alerts.

2. transformers (accepted, not fixable by bump)

CVE-2026-4372 (RCE) and CVE-2026-1839 (malicious-checkpoint code execution) affect transformers 4.57.6 in the analyzer's optional transformers extra. Every spacy-huggingface-pipelines release (0.0.1–0.0.4) caps transformers <5.0.0, but the fixes require >=5.3.0 / >=5.0.0rc3 — no version satisfies both.

Added a documented .trivyignore for the two CVEs/GHSAs.
Already allow-listed for Dependency Review via allow-ghsas in ci.yml (from build: migrate CI and service images from Poetry to uv #2152).
A proper fix (dropping/replacing spacy-huggingface-pipelines to unlock transformers 5.x) is deferred to a follow-up, as it is a functional change to the analyzer.

Note: if the Microsoft Security DevOps action does not honor .trivyignore, the two transformers alerts may still need a one-time dismissal in the code-scanning UI.

Issue reference

N/A

Checklist

I have reviewed the contribution guidelines
I agree to follow this project's Code of Conduct
I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
My code includes unit tests (dependency/CI-only change)
All unit tests and lint checks pass locally
My PR contains documentation updates / additions if required

The analyzer test job runs `poetry install --all-extras` with no committed lock, so Poetry performs a universal resolve of the full optional-dependencies graph on every run. That resolve backtracks for many minutes — effectively hanging — driven by the langextract extra's deep, loosely-bounded transitive tree. Reproduced locally: Poetry >6 min (never completed); uv ~2s for the same 158-package resolution. Switch the test jobs to uv: - Install uv via astral-sh/setup-uv. - Create a per-component venv with `uv venv --seed` (seed keeps pip for the wheel-build step) and expose it via $GITHUB_PATH / $VIRTUAL_ENV. - `uv pip install -e '.<extras>'` plus the test tooling that previously came from the Poetry dev group (uv does not read that group). - Add a job-level `timeout-minutes: 60` (and 30 on the install step) so a bad resolve fails fast instead of hanging. No package metadata changed — pyproject.toml is untouched; this only swaps the CI install tooling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Use "uv run --no-sync" for spaCy model downloads, pytest, and diff-cover instead of exposing the venv on $GITHUB_PATH. "uv sync" builds the project's .venv (resolving the analyzer --all-extras graph in seconds where Poetry's lockless universal resolve hangs), and the subsequent steps execute inside it idiomatically. The --all-extras selection stays in the job matrix, unchanged from main. Wheel-build keeps using the system interpreter, matching the previous Poetry behaviour. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The astral-sh/setup-uv action is not on the org's allowed-actions list, so adding it made the CI workflow fail at startup (startup_failure) before any job ran. Install a pinned uv from PyPI with the already present setup-python instead; this needs no new third-party action and keeps the resolver benefits that motivated the Poetry -> uv switch. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

spaCy's `download` command shells out to `python -m pip install`, but uv-created virtualenvs ship without pip, so runtime/first-use model downloads (e.g. the presidio-cli conftest fetching en_core_web_lg) failed with SystemExit: 1. Install pip alongside the test tooling. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

`python -m build` imports `tomli` on Python < 3.11 to read pyproject.toml, but it is not in the hashed requirements-build.txt (pip-compiled on 3.11+ where tomllib is stdlib). Poetry previously pulled tomli onto the runner as one of its own dependencies, so `pip install --require-hashes` found it already satisfied. uv has no Python dependencies, so the 3.10 wheel-build jobs began failing with "requirements must be pinned with ==". Pre-install a marker-guarded tomli to restore that condition. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The local-build-and-E2E job builds the analyzer/anonymizer/image-redactor images via `docker compose build`. Each Dockerfile ran `poetry install --no-root --only=main -E server`, which triggered the same lockless Poetry backtracking that hangs the test jobs: the analyzer image sat on "Resolving dependencies" for ~40 minutes until the job was cancelled. Switch the dependency install to `uv pip install --system -r pyproject.toml --extra server`, which resolves the same graph in seconds (verified locally: the analyzer install layer completes in ~9s and imports spacy/flask/gunicorn while correctly omitting the project itself, matching --no-root). Drop the now-unused POETRY_VIRTUALENVS_CREATE env and switch `poetry run python install_nlp_models.py` to a direct call. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The Docker images no longer install Poetry (deps come from uv), so the `exec poetry run gunicorn ...` entrypoints crashed at startup with "exec: poetry: not found". The analyzer/anonymizer/image-redactor containers never came up, so every E2E test failed with connection refused on ports 5002/5003. uv installs gunicorn system-wide, so invoke it directly. Verified locally: the anonymizer container now starts healthy and /health returns 200. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Restore the intent of the old POETRY_CACHE_DIR=/mnt/poetry_cache setup for uv: point UV_CACHE_DIR at the runner's large ephemeral disk (/mnt, ~65 GB) so the heavy analyzer --all-extras wheel cache does not fill the smaller root volume. /mnt is root-owned, so a prep step creates the directory and hands it to the runner user. UV_LINK_MODE=copy silences the cross-filesystem hardlink warning, since the project venv lives on the root volume under the checkout while the cache is on /mnt. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Address review feedback that the PR scope grew beyond the test jobs: the CHANGELOG now notes the analyzer/anonymizer/image-redactor image builds and entrypoints also moved from Poetry to uv, and clarifies pyproject.toml metadata is unchanged. Also clarify the ci.yml install-step comment that the wheel-build step deliberately uses the system interpreter (not uv run). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Migrate each CI-tested package's development tooling from Poetry's [tool.poetry.group.dev.dependencies] to the standard PEP 735 [dependency-groups] table, and commit a uv.lock per package. This makes pyproject.toml the single source of truth for dev tooling and lets CI and Docker install a pinned, reproducible dependency graph instead of resolving on every run. The [project] metadata and the poetry-core build backend are unchanged, so wheels/sdists are byte-for-byte identical. Covers presidio-analyzer, presidio-anonymizer, presidio-cli, presidio-image-redactor, presidio-structured. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

CI now runs `uv sync --locked --group dev`, which installs exactly the locked graph and fails if pyproject.toml and uv.lock disagree rather than silently resolving new versions. The dev group provides the test tooling (and pip, which spaCy's model download shells out to), so the manual `uv pip install pytest ...` line is gone. The uv cache is persisted across runs via actions/cache keyed on uv.lock and trimmed with `uv cache prune --ci`. The analyzer/anonymizer/image-redactor images consume the same lockfile with `uv sync --locked --no-default-groups --extra server --no-install-project` into UV_PROJECT_ENVIRONMENT=/usr/local, guaranteeing images use the same dependency graph as CI. Splitting the dependency layer from the source COPY keeps it cached until pyproject/uv.lock change. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The newly committed uv.lock files pin exact versions, so dependency-review now sees two transformers advisories that already exist on main (capped at transformers <5 by spacy-huggingface-pipelines and tracked as Dependabot alerts). They are unrelated to the Poetry->uv migration and will be addressed in separate security PRs, so allow-list the two GHSAs here to keep this PR's dependency review unblocked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@SharonHart

Address review feedback from @SharonHart: - CODEOWNERS: require presidio-administrators approval for **/uv.lock, the same as **/pyproject.toml, so lockfile changes get dependency review. - copilot-instructions: switch the local dev commands to uv and add an explicit rule that editing pyproject.toml dependencies requires regenerating and committing that package's uv.lock in the same change, since CI installs with `uv sync --locked` and fails on drift. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Address review feedback: - ci.yml: use ${{ env.UV_CACHE_DIR }} for the cache action path instead of hardcoding /mnt/uv-cache, so the cache and uv stay in sync if the dir changes. - copilot-instructions: update the "Technology Stack" bullet to name uv as the dependency/install tool (poetry-core retained only as build backend), matching the uv-based dev commands already documented. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

presidio-structured's lock resolved cryptography 46.0.7, which carries the vulnerable OpenSSL bundled in older cryptography wheels (GHSA-537c-gmf6-5ccf). Add a uv constraint-dependencies pin forcing cryptography>=48.0.1; the lock now uses 49.0.0. The published presidio-anonymizer 2.2.363 caps cryptography <47, so the resolver falls back to presidio-anonymizer 2.2.362 (which allows the patched line) in structured's lock. That is a harmless CI-lock change; the repo's own anonymizer already declares cryptography >=48.0.1 and a future anonymizer release will lift the cap. Structured's test suite passes (28 passed). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

The analyzer's optional transformers extra pulls spacy-huggingface-pipelines, whose every release caps transformers <5.0.0, while CVE-2026-4372 (RCE) and CVE-2026-1839 (malicious-checkpoint code execution) are only fixed in transformers 5.3.0 / 5.0.0rc3. No version satisfies both, so add a documented .trivyignore accepting the two advisories until the capping dependency is dropped/replaced in a follow-up. These are already allow-listed for the dependency-review job via allow-ghsas in ci.yml. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Trim the verbose explanatory comments in the test job to 1-3 lines each, and drop the dependency-review `allow-ghsas` entry for the transformers advisories. That suppression is a security concern rather than part of the Poetry->uv migration, so it moves to the stacked security PR alongside the .trivyignore. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…t' into ronshakutai/fix-uv-lock-security

Moved here from the base uv-migration PR: the transformers advisories can't be remediated by a version bump (spacy-huggingface-pipelines caps transformers <5), so allow-list them for dependency-review alongside the .trivyignore that covers the Trivy scan. Tracked for a real fix (dropping spacy-huggingface-pipelines) in a follow-up. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

…into ronshakutai/fix-uv-lock-security

github-actions · 2026-07-13T05:58:56Z

Coverage report (presidio-anonymizer)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-anonymizer/presidio_anonymizer
anonymizer_engine.py
presidio-anonymizer/presidio_anonymizer/operators
custom.py
Project Total

_{This report was generated by python-coverage-comment-action}

github-actions · 2026-07-13T05:59:05Z

Coverage report (presidio-structured)

This PR does not seem to contain any modification to coverable code.

Copilot

Pull request overview

This PR updates dependency lock/scanning configuration to address CI-reported security findings: it forces presidio-structured to resolve a patched cryptography version, and documents/accepts currently-unfixable transformers advisories by adding a Trivy ignore list and allow-listing the corresponding GHSAs for Dependency Review.

Changes:

Constrain cryptography in presidio-structured via tool.uv.constraint-dependencies, resulting in cryptography 49.0.0 in presidio-structured/uv.lock.
Add a root .trivyignore to suppress specific transformers CVE/GHSA findings that cannot be remediated due to upstream version caps.
Allow-list the same transformers GHSAs in the Dependency Review workflow configuration.

Reviewed changes

Copilot reviewed 3 out of 4 changed files in this pull request and generated no comments.

File	Description
`presidio-structured/uv.lock`	Applies the `cryptography>=48.0.1` constraint in the resolved lock graph (updates `cryptography` and related resolution).
`presidio-structured/pyproject.toml`	Adds `[tool.uv] constraint-dependencies` to enforce a patched `cryptography` line without making it a direct dependency.
`.trivyignore`	Documents and suppresses the two unfixable `transformers` advisories for Trivy-based scanning.
`.github/workflows/ci.yml`	Allow-lists the corresponding `transformers` GHSAs for the Dependency Review action to keep CI green.

github-actions · 2026-07-13T06:00:18Z

Coverage report (presidio-cli)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-cli/presidio_cli
cli.py
Project Total

_{This report was generated by python-coverage-comment-action}

github-actions · 2026-07-13T06:01:32Z

Coverage report (presidio-image-redactor)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-image-redactor/presidio_image_redactor
dicom_image_pii_verify_engine.py
document_intelligence_ocr.py
image_analyzer_engine.py
Project Total

_{This report was generated by python-coverage-comment-action}

github-actions · 2026-07-13T06:12:53Z

Coverage report (presidio-analyzer)

Click to see where and how coverage changed

File	Statements	Missing	Coverage	Coverage (new stmts)	Lines missing
presidio-analyzer/presidio_analyzer
analyzer_engine.py
entity_recognizer.py
presidio-analyzer/presidio_analyzer/input_validation
schemas.py
yaml_recognizer_models.py
presidio-analyzer/presidio_analyzer/nlp_engine
__init__.py
nlp_engine_provider.py
presidio-analyzer/presidio_analyzer/predefined_recognizers
__init__.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific
__init__.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/australia
au_abn_recognizer.py
au_acn_recognizer.py
au_medicare_recognizer.py
au_tfn_recognizer.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/finland
fi_personal_identity_code_recognizer.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/germany
de_bsnr_recognizer.py
de_id_card_recognizer.py
de_lanr_recognizer.py
de_passport_recognizer.py
de_social_security_recognizer.py
de_tax_id_recognizer.py
de_vat_id_recognizer.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/india
in_vehicle_registration_recognizer.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/country_specific/poland
pl_pesel_recognizer.py
presidio-analyzer/presidio_analyzer/predefined_recognizers/generic
phone_recognizer.py
presidio-analyzer/presidio_analyzer/recognizer_registry
recognizer_registry.py
recognizers_loader_utils.py
Project Total

The report is truncated to 25 files out of 69. To see the full report, please visit the workflow summary page.

_{This report was generated by python-coverage-comment-action}

SharonHart · 2026-07-13T07:00:53Z

Will be handeled by bumping anonymizer in presidio-structured

RonShakutai and others added 20 commits July 12, 2026 11:04

Potential fix for pull request finding

1bae43e

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Merge remote-tracking branch 'origin/ronshakutai/fix-ci-poetry-timeou…

afe023a

…t' into ronshakutai/fix-uv-lock-security

RonShakutai mentioned this pull request Jul 13, 2026

build: migrate CI and service images from Poetry to uv #2152

Merged

6 tasks

Base automatically changed from ronshakutai/fix-ci-poetry-timeout to main July 13, 2026 05:42

Merge branch 'main' of https://github.com/data-privacy-stack/presidio …

bbb5545

…into ronshakutai/fix-uv-lock-security

Copilot AI review requested due to automatic review settings July 13, 2026 05:57

Copilot started reviewing on behalf of RonShakutai July 13, 2026 05:58 View session

Copilot AI reviewed Jul 13, 2026

View reviewed changes

SharonHart closed this Jul 13, 2026

SharonHart deleted the ronshakutai/fix-uv-lock-security branch July 13, 2026 07:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(security): patch cryptography in structured lock; accept unfixable transformers advisories#2158

fix(security): patch cryptography in structured lock; accept unfixable transformers advisories#2158
RonShakutai wants to merge 21 commits into
mainfrom
ronshakutai/fix-uv-lock-security

RonShakutai commented Jul 12, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

github-actions Bot commented Jul 13, 2026

Uh oh!

SharonHart commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

RonShakutai commented Jul 12, 2026

Change Description

1. cryptography (real fix)

2. transformers (accepted, not fixable by bump)

Issue reference

Checklist

Uh oh!

github-actions Bot commented Jul 13, 2026

Coverage report (presidio-anonymizer)

Uh oh!

github-actions Bot commented Jul 13, 2026

Coverage report (presidio-structured)

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

github-actions Bot commented Jul 13, 2026

Coverage report (presidio-cli)

Uh oh!

github-actions Bot commented Jul 13, 2026

Coverage report (presidio-image-redactor)

Uh oh!

github-actions Bot commented Jul 13, 2026

Coverage report (presidio-analyzer)

Uh oh!

SharonHart commented Jul 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants