Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ All notable changes to this project will be documented in this file.
## [unreleased]

### Analyzer

#### Added
- Czech PII recognizers for `CZ_BIRTH_NUMBER`, `CZ_BANK_ACCOUNT`, `CZ_ID_CARD`, `CZ_PASSPORT`, and `CZ_DRIVER_LICENSE`, plus Czech `DATE_TIME` date coverage (`CzDateRecognizer`); all are disabled by default. Includes a Czech language support recipe (`docs/recipes/czech-language-support`)

#### Fixed
- Language model recognizers (`BasicLangExtractRecognizer`, `AzureOpenAILangExtractRecognizer`) configured in a recognizer registry YAML now honour `config_path` (and other recognizer-specific kwargs). Previously these entries were validated by the strict `PredefinedRecognizerConfig` schema, which has no `config_path` field and does not allow extra keys, so `config_path` was silently dropped and the recognizer fell back to its bundled default model configuration. Added a `LangExtractRecognizerConfig` model (`extra="allow"`) and registered both recognizer class names in `CONFIG_MODEL_MAP`.
- `BasicLangExtractRecognizer` now honours values under `langextract.model.provider.language_model_params` (including `timeout` and `num_ctx`). Previously these were silently dropped because `langextract.extract()` ignores its `language_model_params` argument when a pre-built `ModelConfig` is passed via `config=`, causing Ollama-backed recognizers to fall back to langextract's 120s default regardless of the configured timeout. The recognizer now merges `language_model_params` into `ModelConfig.provider_kwargs`, which is the path that reaches the provider constructor. Explicit entries under `provider.kwargs:` still take precedence. Also fixed a `TypeError` when `kwargs:` or `language_model_params:` is `null` in the YAML. (#1943, Thanks @lsternlicht)
Expand Down
183 changes: 183 additions & 0 deletions docs/recipes/czech-language-support/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,183 @@
# Czech Language Support
Comment thread
santiagotri marked this conversation as resolved.

> **Domain**: Finance, Legal, Public Administration, General
> **Data Type**: Czech-language free text (contracts, bank correspondence, ID documents, customer communication)
> **Goal**: Detect Czech PII and sensitive identifiers using Presidio's built-in Czech recognizers with a bilingual spaCy NLP engine

## Overview

**Domain**: Finance / Legal / Public Administration
**Data Type**: Czech-language documents
**Goal**: Detect Czech PII — including the rodné číslo (birth number), domestic
bank account numbers, official document numbers and Czech-notation dates —
alongside the English model so that bilingual (EN + CS) documents are
handled correctly.

This recipe provides:

- `spacy_en_cs.yaml` — a ready-to-use NLP engine configuration that loads both
`en_core_web_lg` and the multilingual `xx_ent_wiki_sm` (spaCy publishes no
Czech-specific pretrained pipeline)
- An overview of all Czech-specific recognizers available in Presidio

## Quick Start

### Prerequisites

```bash
pip install presidio-analyzer presidio-anonymizer
python -m spacy download en_core_web_lg
python -m spacy download xx_ent_wiki_sm
```

### Sample Data

```python
sample_text = (
"Jmenuji se Jan Novák a narodil jsem se dne 12. dubna 1985. "
"Moje rodné číslo je 850412/0003. "
"Kontaktovat mě můžete na telefonním čísle +420 601 234 567. "
"Moje číslo občanského průkazu je 123456789 "
"a číslo cestovního pasu je AB123456. "
"Bankovní účet mám vedený pod číslem 19-123456789/0800. "
"IBAN účtu je CZ6508000000192000145399."
)
```

### Basic Configuration

```python
from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.nlp_engine import NlpEngineProvider
from presidio_analyzer.predefined_recognizers import (
CzBankAccountRecognizer,
CzBirthNumberRecognizer,
CzDateRecognizer,
CzDriverLicenseRecognizer,
CzIdCardRecognizer,
CzPassportRecognizer,
)
from presidio_anonymizer import AnonymizerEngine

# Load the bilingual EN + CS spaCy configuration
provider = NlpEngineProvider(conf_file="spacy_en_cs.yaml")
nlp_engine = provider.create_engine()

analyzer = AnalyzerEngine(
nlp_engine=nlp_engine,
supported_languages=["en", "cs"],
)

# The Czech recognizers ship disabled by default (upstream convention for
# non-default languages); register them for the Czech pipeline explicitly.
for recognizer in (
CzBirthNumberRecognizer(),
CzBankAccountRecognizer(),
CzIdCardRecognizer(),
CzPassportRecognizer(),
CzDriverLicenseRecognizer(),
CzDateRecognizer(),
):
analyzer.registry.add_recognizer(recognizer)

anonymizer = AnonymizerEngine()

# Analyze Czech text
results = analyzer.analyze(text=sample_text, language="cs")
anonymized = anonymizer.anonymize(text=sample_text, analyzer_results=results)

print(anonymized.text)
```

Alternatively, enable the recognizers declaratively by copying
`default_recognizers.yaml` and flipping `enabled: true` on the `Cz*` entries.

## Approach

Presidio ships pattern-based recognizers for 5 Czech entity types plus Czech
`DATE_TIME` coverage (see table below). Each recognizer targets a single
entity, uses lookaround-anchored regex patterns with base confidence between
0.1 and 0.6, and relies on:

1. **Context words** (inflected Czech terminology near the match — the
enhancer matches surface forms, so declensions like "účtu", "průkazu"
are listed explicitly) to boost confidence
2. **Check digit validation** where the official specification defines one
(rodné číslo mod 11, bank account weighted mod 11)

Identifiers that regularly collide with `PHONE_NUMBER` (rodné číslo such as
`850412/0003`, bank accounts such as `19-123456789/0800`) rely on the
combination of a higher base score and context boosting to win score-based
conflict resolution against the phone recognizer's flat 0.4 score.

The multilingual `xx_ent_wiki_sm` model adds named-entity recognition for
PERSON, LOCATION and ORGANIZATION on top of the pattern recognizers. Czech
dates ("12. dubna 1985") are covered by `CzDateRecognizer` because no Czech
NER model emitting DATE labels is available.

### Supported Czech Entities

| Entity | Name | Check digit |
|--------|------|-------------|
| `CZ_BIRTH_NUMBER` | Rodné číslo | ✅ mod 11 (§ 13 z. č. 133/2000 Sb.) |
| `CZ_BANK_ACCOUNT` | Číslo bankovního účtu | ✅ weighted mod 11 (vyhláška ČNB č. 169/2011 Sb.) |
| `CZ_ID_CARD` | Číslo občanského průkazu | – |
| `CZ_PASSPORT` | Číslo cestovního pasu | – |
| `CZ_DRIVER_LICENSE` | Číslo řidičského průkazu | – |
| `DATE_TIME` | Czech date notation (via `CzDateRecognizer`) | – |

## Results

Formal evaluation against a labelled Czech dataset has not yet been performed.
To benchmark this recipe follow the [Presidio Research evaluation workflow](https://github.com/data-privacy-stack/presidio-research/blob/master/notebooks/4_Evaluate_Presidio_Analyzer.ipynb):

1. Generate synthetic Czech text with the [data generator](https://github.com/data-privacy-stack/presidio-research/blob/master/notebooks/1_Generate_data.ipynb)
2. Configure the analyzer with `spacy_en_cs.yaml`
3. Run the evaluator and report precision / recall / F₂ / latency

**Precision**: TBD
**Recall**: TBD
**F₂ Score**: TBD
**Latency**: TBD

### Key Findings

- Recognizers with check digit validation (rodné číslo, bank account) achieve
very low false-positive rates on checksum-valid inputs, and deliberately
keep checksum-failing but date-plausible matches at the base pattern score:
many circulating rodná čísla predate the strict mod 11 rule, so context
words decide those cases instead of a hard reject.
- Recognizers without a checksum (ID card, passport, driving licence)
rely heavily on context words; setting `score_threshold=0.4` filters
context-free digit strings effectively.
- Czech legal citations (`262/2006 Sb.`, `89/2012 Sb.`) share the `NNN/YYYY`
shape with prefix-less bank accounts and are ubiquitous in legal text; the
bank account recognizer invalidates prefix-less matches with a numerator of
up to 3 digits and a year-like 1900–2099 "bank code", and never promotes a
checksum pass over a year-like code the ČNB did not issue.
- `xx_ent_wiki_sm` ships without a lemmatizer, so the context enhancer
compares surface forms. The bundled context word lists therefore contain
the inflected forms that actually occur next to each identifier.

## Tips for Others

- **Set `score_threshold`** to 0.4–0.5 for production use to filter out
low-confidence pattern-only matches from context-free digit strings.
- **Use the `entities` parameter** to limit detection to the entity types
relevant to your domain (e.g. only `CZ_BIRTH_NUMBER` and `CZ_BANK_ACCOUNT`
in banking correspondence).
- **Higher-quality Czech NER**: for better PERSON/LOCATION recall, switch to
the `TransformersNlpEngine` with a Czech NER model from the HuggingFace hub
(e.g. [`richielo/small-e-czech-finetuned-ner-wikiann`](https://huggingface.co/richielo/small-e-czech-finetuned-ner-wikiann),
a Czech ELECTRA model fine-tuned on WikiANN) and keep the same
`model_to_presidio_entity_mapping`. Stanza currently provides no Czech NER
package either, so transformers is the recommended upgrade path.
- **Full street addresses** ("Václavské náměstí 123/45, 110 00 Praha 1") are
only as good as the NER model's LOCATION spans; generic models often split
them into street and city fragments. Add a custom address recognizer if
your domain requires whole-address redaction.

---

**Author**: Data Privacy Stack contributors
**Date**: 2026-07-11
30 changes: 30 additions & 0 deletions docs/recipes/czech-language-support/spacy_en_cs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
nlp_engine_name: spacy
models:
-
lang_code: en
model_name: en_core_web_lg
-
# spaCy publishes no Czech-specific pretrained pipeline; the official
# multilingual WikiNER model provides PER/LOC/ORG NER for Czech text.
# See the recipe README for a higher-quality transformers alternative.
lang_code: cs
model_name: xx_ent_wiki_sm

ner_model_configuration:
model_to_presidio_entity_mapping:
PER: PERSON
PERSON: PERSON
NORP: NRP
FAC: LOCATION
LOC: LOCATION
LOCATION: LOCATION
GPE: LOCATION
ORG: ORGANIZATION
ORGANIZATION: ORGANIZATION
DATE: DATE_TIME
TIME: DATE_TIME

low_confidence_score_multiplier: 0.4
low_score_entity_names:
- ORG
- ORGANIZATION
13 changes: 13 additions & 0 deletions docs/supported_entities.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,19 @@ For more information, refer to the [adding new recognizers documentation](analyz
| DE_HANDELSREGISTER | German Handelsregisternummer: commercial register number with HRA (sole traders / partnerships) or HRB (corporations) prefix followed by 1–6 digits. HRA entries directly identify natural persons (sole traders). Legal basis: §§ 9, 14 HGB, HRV. | Pattern match and context |
| DE_PLZ | German Postleitzahl (postal code): 5-digit code in the range 01001–99998. Constitutes personal data in combination with other address fields (DSGVO Art. 4 Nr. 1). **High false-positive risk** – only reliable with address-context words present; base confidence is 0.05. Legal basis: DSGVO Art. 4 Nr. 1. | Pattern match and context (context required for actionable results) |

### Czechia

| Entity Type | Description | Detection Method |
| --- | --- | --- |
| CZ_BIRTH_NUMBER | Czech rodné číslo (birth number): the national identification number encoding date of birth and sex, formatted YYMMDD/SSS (until 1953) or YYMMDD/SSSS (since 1954), with or without the slash. Legal basis: § 13 zákona č. 133/2000 Sb. | Pattern match, context and checksum (mod 11, incl. the 1954–1985 historical exception) |
| CZ_BANK_ACCOUNT | Czech domestic bank account number in the national format `[prefix-]number/bank code` (e.g. 19-2000145399/0800). Legal basis: vyhláška ČNB č. 169/2011 Sb. | Pattern match, context and checksum (weighted mod 11 on prefix and account number) |
| CZ_ID_CARD | Czech občanský průkaz (identity card) number: 9 digits on machine readable cards and the current eOP. **Context words required for actionable results** — base confidence is 0.2. Legal basis: zákon č. 269/2021 Sb. | Pattern match and context |
| CZ_PASSPORT | Czech cestovní pas (passport) number: 2 letters + 6–7 digits (older series) or 7–8 digits (biometric series). Legal basis: zákon č. 329/1999 Sb. | Pattern match and context |
| CZ_DRIVER_LICENSE | Czech řidičský průkaz (driving licence) number: 2 letters + 6–9 digits. Legal basis: § 103 zákona č. 361/2000 Sb. | Pattern match and context |

Czech-language pipelines additionally get `DATE_TIME` coverage for Czech date
notation ("12. dubna 1985", "12. 4. 1985") through the `CzDateRecognizer`.

### Medical / Clinical

Detected using the `MedicalNERRecognizer` (requires the `transformers` extra). Uses the [blaze999/Medical-NER](https://huggingface.co/blaze999/Medical-NER) model by default.
Expand Down
43 changes: 43 additions & 0 deletions presidio-analyzer/presidio_analyzer/conf/default_recognizers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -481,6 +481,49 @@ recognizers:
enabled: false
country_code: de

# Czechia recognizers (disabled by default, enable for Czech-language pipelines)
- name: CzBirthNumberRecognizer
supported_languages:
- cs
type: predefined
enabled: false
country_code: cz

- name: CzBankAccountRecognizer
supported_languages:
- cs
type: predefined
enabled: false
country_code: cz

- name: CzIdCardRecognizer
supported_languages:
- cs
type: predefined
enabled: false
country_code: cz

- name: CzPassportRecognizer
supported_languages:
- cs
type: predefined
enabled: false
country_code: cz

- name: CzDriverLicenseRecognizer
supported_languages:
- cs
type: predefined
enabled: false
country_code: cz

- name: CzDateRecognizer
supported_languages:
- cs
type: predefined
enabled: false
country_code: cz

- name: BasicLangExtractRecognizer
supported_languages:
- en
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,20 @@
# Canada recognizers
from .country_specific.canada.ca_sin_recognizer import CaSinRecognizer

# Czechia recognizers
from .country_specific.czechia.cz_bank_account_recognizer import (
CzBankAccountRecognizer,
)
from .country_specific.czechia.cz_birth_number_recognizer import (
CzBirthNumberRecognizer,
)
from .country_specific.czechia.cz_date_recognizer import CzDateRecognizer
from .country_specific.czechia.cz_driver_license_recognizer import (
CzDriverLicenseRecognizer,
)
from .country_specific.czechia.cz_id_card_recognizer import CzIdCardRecognizer
from .country_specific.czechia.cz_passport_recognizer import CzPassportRecognizer

# Finland recognizers
from .country_specific.finland.fi_personal_identity_code_recognizer import (
FiPersonalIdentityCodeRecognizer,
Expand Down Expand Up @@ -256,6 +270,13 @@
"NgNinRecognizer",
"NgVehicleRegistrationRecognizer",
"MedicalNERRecognizer",
# Czechia recognizers
"CzBirthNumberRecognizer",
"CzBankAccountRecognizer",
"CzDateRecognizer",
"CzDriverLicenseRecognizer",
"CzIdCardRecognizer",
"CzPassportRecognizer",
# Germany recognizers
"DeTaxIdRecognizer",
"DeTaxNumberRecognizer",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
"""Czechia-specific recognizers package."""
Loading