feat(analyzer): add recognizer-level threshold config by rodboev · Pull Request #2116 · data-privacy-stack/presidio

rodboev · 2026-06-28T04:08:46Z

Change Description

Adds recognizer-level score thresholds in presidio-analyzer. Each recognizer entry can set a default threshold and entity-specific overrides, while AnalyzerEngine keeps final filtering after context enhancement and before duplicate removal.

recognizer_registry:
  recognizers:
    - name: CreditCardRecognizer
      type: predefined
      score_thresholds:
        default: 0.4
        CREDIT_CARD: 0.7

RecognizerRegistryProvider validates and attaches normalized thresholds to each recognizer instance. The analyzer resolves the producing recognizer by recognizer_identifier, so same-name recognizers, programmatic recognizers, and request-only ad hoc recognizers retain independent policies.

Precedence is explicit request score_threshold, entity override, recognizer default, then default_score_threshold. The former top-level recognizer_score_thresholds key is removed because this PR has not merged.

This updates the analyzer and registry configuration docs, no-code example, default YAML examples, and changelog.

Issue reference

Fixes #1572

Tests

poetry run pytest tests/test_entity_recognizer.py tests/test_yaml_recognizer_models.py tests/test_recognizers_loader_utils.py tests/test_recognizer_registry_provider.py tests/test_recognizer_registry.py tests/test_configuration_validator.py tests/test_analyzer_engine.py tests/test_analyzer_engine_provider.py -q with 303 passed and 1 skipped
poetry run ruff check presidio_analyzer/score_thresholds.py presidio_analyzer/entity_recognizer.py presidio_analyzer/analyzer_engine.py presidio_analyzer/analyzer_engine_provider.py presidio_analyzer/input_validation/schemas.py presidio_analyzer/input_validation/yaml_recognizer_models.py presidio_analyzer/recognizer_registry/recognizer_registry_provider.py presidio_analyzer/recognizer_registry/recognizers_loader_utils.py presidio_analyzer/recognizer_registry/recognizer_registry.py tests/test_entity_recognizer.py tests/test_yaml_recognizer_models.py tests/test_recognizers_loader_utils.py tests/test_recognizer_registry_provider.py tests/test_recognizer_registry.py tests/test_configuration_validator.py tests/test_analyzer_engine.py tests/test_analyzer_engine_provider.py

Note on CHANGELOG

Updated CHANGELOG.md under [unreleased], Analyzer, Added.

Checklist

I have reviewed the contribution guidelines
I agree to follow this project's Code of Conduct
I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
My code includes unit tests
All unit tests and lint checks pass locally
My PR contains documentation updates / additions if required

rodboev · 2026-06-28T04:09:00Z

@microsoft-github-policy-service agree

Copilot

Pull request overview

Adds support in presidio-analyzer for configuring score thresholds per recognizer (and optionally per entity within that recognizer) via YAML, while preserving the existing global default_score_threshold / request-level score_threshold behavior.

Changes:

Introduces recognizer_score_thresholds configuration, applies it during result filtering (before duplicate collapse) when no request-level score_threshold is provided.
Adds validation/normalization for the new configuration shape (including numeric shorthand) and expands unit test coverage for precedence and error cases.
Updates analyzer configuration docs, the no-code tutorial, and the root changelog to document the new option.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
presidio-analyzer/presidio_analyzer/analyzer_engine.py	Adds recognizer/entity-specific threshold support and applies filtering before deduplication.
presidio-analyzer/presidio_analyzer/analyzer_engine_provider.py	Loads `recognizer_score_thresholds` from analyzer YAML and passes it into `AnalyzerEngine`.
presidio-analyzer/presidio_analyzer/input_validation/schemas.py	Validates and normalizes `recognizer_score_thresholds` in analyzer configuration validation.
presidio-analyzer/presidio_analyzer/conf/default_analyzer_full.yaml	Documents the new YAML option with a commented example.
presidio-analyzer/tests/test_analyzer_engine.py	Adds deterministic tests covering precedence, shorthand, invalid values, and dedupe ordering.
presidio-analyzer/tests/test_analyzer_engine_provider.py	Verifies provider passes/normalizes thresholds from YAML and that defaults remain unchanged otherwise.
presidio-analyzer/tests/test_configuration_validator.py	Adds validator tests for valid shorthand + invalid threshold structures/types/ranges.
docs/tutorial/08_no_code.md	Updates no-code YAML example and explains when to use recognizer-level thresholds.
docs/analyzer/analyzer_engine_provider.md	Documents the new configuration key and provides a YAML example.
CHANGELOG.md	Adds an Unreleased entry documenting the new analyzer YAML capability.

Copilot

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

)

omri374

Thanks! Looks really good. A few minor and quickly fixable comments

omri374 · 2026-07-13T09:13:51Z

+
+
+def validate_score_threshold(threshold: Any) -> float:
+    """Validate a score threshold without coercing its input type."""


please align the docstring format to the rest of the repo (add :param: threshold)

Do we have to keep threshold as Any? can we use a more specific type?

omri374 · 2026-07-13T09:14:18Z

+    return threshold
+
+
+def normalize_score_thresholds(score_thresholds: Any) -> Dict[str, float]:


Same comments as above

omri374 · 2026-07-13T09:15:00Z

        version: str = "0.0.1",
        context: Optional[List[str]] = None,
        country_code: Optional[str] = None,
+        score_thresholds: Any = None,


Can we provide a more specific type? I assume it's dict[str, float]?

omri374 · 2026-07-13T09:15:38Z

-        self, results: List[RecognizerResult], score_threshold: float = None
+        self,
+        results: List[RecognizerResult],
+        score_threshold: float = None,


Suggested change

score_threshold: float = None,

score_threshold: Optional[float] = None,

omri374 · 2026-07-13T09:36:43Z

 """
 ```

+Each recognizer can set a `default` cutoff and entity-specific overrides in `score_thresholds`. An explicit request threshold takes priority, followed by an entity override, the recognizer default, and the analyzer's `default_score_threshold`.


This is an important implementation detail. If we pass score_threshold, then the recognizer-level thresholds are bypassed completely. Please make this more explicit ("explicit request threshold" might not be clear) and put this in the docs, not just the tutorial.

here also, please use "threshold" instead of "cutoff"

omri374 · 2026-07-13T09:40:24Z

        """
        if score_threshold is None:
-            score_threshold = self.default_score_threshold
+            recognizers_by_id = {


if id is missing for some reason, it would not pick up the threshold. Consider adding a debug log

omri374 · 2026-07-13T09:41:53Z

+    @property
+    def score_thresholds(self) -> Dict[str, float]:
+        """Return a defensive copy of this recognizer's score thresholds."""
+        return self._score_thresholds.copy()


This is copied on every result. Is it necessary?

omri374 · 2026-07-13T09:45:45Z

  - `supported_entity`: the detected entity associated by the recognizer.
  - `deny_list`: A list of words to detect, in case the recognizer uses a predefined list of words.
  - `deny_list_score`: confidence score for a term identified using a deny-list.
+  - `score_thresholds`: optional score cutoffs for this recognizer. Use `default` as the recognizer-wide cutoff and entity names for overrides. An explicit request threshold takes priority, followed by an entity override, this recognizer default, and the analyzer's `default_score_threshold`.


please use "threshold" instead of "cutoff" to keep the jargon similar to the rest of the code base

omri374 · 2026-07-13T09:46:27Z

        :param entities: List of PII entities that should be looked for in the text.
        If entities=None then all entities are looked for.
        :param correlation_id: cross call ID for this request
        :param score_threshold: A minimum value for which


please update this threshold with the new logic

omri374 · 2026-07-13T09:48:23Z

+    assert scores == {"URL": 0.75, "DATE_TIME": 0.9}
+
+
+def test_explicit_score_threshold_overrides_config():


please make sure we are covering multiple competing thresholds scenarios.

Copilot AI review requested due to automatic review settings June 28, 2026 04:08

github-actions Bot added the external label Jun 28, 2026

Copilot started reviewing on behalf of rodboev June 28, 2026 04:09 View session

Copilot AI reviewed Jun 28, 2026

View reviewed changes

SharonHart requested review from Copilot and omri374 July 9, 2026 11:45

Copilot started reviewing on behalf of SharonHart July 9, 2026 11:45 View session

Copilot AI reviewed Jul 9, 2026

View reviewed changes

Comment thread presidio-analyzer/presidio_analyzer/input_validation/schemas.py Outdated

Comment thread presidio-analyzer/presidio_analyzer/input_validation/schemas.py Outdated

Comment thread presidio-analyzer/presidio_analyzer/analyzer_engine.py Outdated

rodboev added 8 commits July 9, 2026 13:22

Allow per-recognizer analyzer tuning from YAML

3c206f0

Keep boolean flags from slipping into threshold config

e458cd4

Keep analyzer docs aligned with the new threshold contract

2c5cb34

Keep direct threshold config aligned with YAML validation

08da64e

Preserve lenient recognizers when duplicate spans collide

5de75b8

Keep threshold validation errors tied to the config key

21606fc

Keep constructor threshold hints aligned with shared validation

94b050b

fix(analyzer): reject boolean score thresholds

adac24d

rodboev force-pushed the pr/recognizer-threshold-config branch from 9ebc0e8 to adac24d Compare July 9, 2026 17:29

omri374 reviewed Jul 9, 2026

View reviewed changes

Comment thread docs/analyzer/analyzer_engine_provider.md Outdated

rodboev and others added 2 commits July 10, 2026 02:02

fix(analyzer): move thresholds onto recognizers (data-privacy-stack#1572

2c49182

)

Merge branch 'main' into pr/recognizer-threshold-config

0bc3e87

omri374 reviewed Jul 13, 2026

View reviewed changes



		def validate_score_threshold(threshold: Any) -> float:
		"""Validate a score threshold without coercing its input type."""

		return threshold


		def normalize_score_thresholds(score_thresholds: Any) -> Dict[str, float]:

	score_threshold: float = None,
	score_threshold: Optional[float] = None,

		assert scores == {"URL": 0.75, "DATE_TIME": 0.9}


		def test_explicit_score_threshold_overrides_config():

Uh oh!

Conversation

rodboev commented Jun 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Change Description

Issue reference

Tests

Note on CHANGELOG

Checklist

Uh oh!

rodboev commented Jun 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

omri374 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rodboev commented Jun 28, 2026 •

edited

Loading