fix(analyzer): honour config_path for LangExtract recognizers in YAML registry#2151
Closed
RonShakutai wants to merge 2 commits into
Closed
Conversation
… registry LM recognizers (BasicLangExtractRecognizer, AzureOpenAILangExtractRecognizer) configured via a recognizer registry YAML silently ignored config_path: the strict PredefinedRecognizerConfig schema has no config_path field and forbids extras, so Pydantic dropped it and the recognizer fell back to its bundled default model configuration. Add a LangExtractRecognizerConfig model (extra=allow, explicit config_path field) mirroring the existing HuggingFace/GLiNER configs, and register both LM recognizer class names in CONFIG_MODEL_MAP so config_path (and other recognizer-specific kwargs) survive validation and reach the constructor. Adds regression tests covering config_path preservation via both the model and the full ConfigurationValidator registry path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ution - Add job-level timeout of 60 minutes to test job - Add step-level timeout of 30 minutes to Install dependencies step - Add --no-interaction --no-directory flags to poetry install command This fixes the issue where CI hangs indefinitely during dependency resolution when Poetry encounters complex dependency graphs (e.g., with --all-extras)
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.
Fixes YAML recognizer-registry validation so LangExtract-based LM recognizers preserve config_path and other extra kwargs, preventing silent fallback to default bundled model configs.
Changes:
- Added
LangExtractRecognizerConfig(extra="allow"+ explicitconfig_path) and mapped LangExtract recognizer class names inCONFIG_MODEL_MAP. - Added regression tests asserting
config_pathsurvives full registry validation for both basic and Azure variants. - Updated CI workflow timeouts and dependency installation flags; updated changelog entry.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| presidio-analyzer/tests/test_yaml_recognizer_models.py | Adds regression tests for config_path preservation and correct config model selection. |
| presidio-analyzer/presidio_analyzer/input_validation/yaml_recognizer_models.py | Introduces LangExtractRecognizerConfig and registers it for LangExtract recognizers in CONFIG_MODEL_MAP. |
| CHANGELOG.md | Documents the fix for YAML config_path handling for LM recognizers. |
| .github/workflows/ci.yml | Adds job/step timeouts and adjusts poetry install flags. |
Comment on lines
+9
to
+10
| - Language model recognizers (`BasicLangExtractRecognizer`, `AzureOpenAILangExtractRecognizer`) configured in a recognizer registry YAML now honour `config_path` (and other recognizer-specific kwargs). Previously these entries were validated by the strict `PredefinedRecognizerConfig` schema, which has no `config_path` field and does not allow extra keys, so `config_path` was silently dropped and the recognizer fell back to its bundled default model configuration. Added a `LangExtractRecognizerConfig` model (`extra="allow"`) and registered both recognizer class names in `CONFIG_MODEL_MAP`. | ||
| - `BasicLangExtractRecognizer` now honours values under `langextract.model.provider.language_model_params` (including `timeout` and `num_ctx`). Previously these were silently dropped because `langextract.extract()` ignores its `language_model_params` argument when a pre-built `ModelConfig` is passed via `config=`, causing Ollama-backed recognizers to fall back to langextract's 120s default timeout regardless of the configured timeout. The recognizer now merges `language_model_params` into `ModelConfig.provider_kwargs`, which is the path that reaches the provider constructor. Explicit entries under `provider.kwargs:` still take precedence. Also fixed a `TypeError` when `kwargs:` or `language_model_params:` is `null` in the YAML. (#1943, Thanks @lsternlicht) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Description
LM recognizers (
BasicLangExtractRecognizer,AzureOpenAILangExtractRecognizer) configured via a recognizer-registry YAML now honourconfig_path.Bug: these class names were missing from
CONFIG_MODEL_MAP, so their YAML entry was validated by the strictPredefinedRecognizerConfig(noconfig_pathfield, noextra="allow"). Pydantic silently droppedconfig_path, and the recognizer fell back to its bundled default model (qwen2.5:1.5b) instead of the configured one.Fix: add a
LangExtractRecognizerConfig(extra="allow"+ explicitconfig_path) mirroring the existing HuggingFace/GLiNER configs, and register both class names inCONFIG_MODEL_MAP. Nowconfig_pathsurvives validation and reaches the constructor.Adds 3 regression tests. Verified end-to-end: an Ollama recognizer now builds with the configured model (e.g.
llama3.2:3b).Issue reference
N/A
Checklist