Story 1.1 – Design fallback spec

[sidmohan0]

• Enumerate MVP entities (email, phone, SSN, credit-card, IPv4/6, DOB, ZIP).
• Define return schema (same Pydantic model spaCy uses).

[sidmohan0]

Story 1.1

---

1. Entity menu (MVP for 4.1)

Label	Scope	Regex sketch	Notes
EMAIL	RFC 5322 subset	[\w.+-]+@[\w-]+.[\w.-]{2,}	Good enough for 99 % of mail; avoids huge RFC monsters. (Regex validation of email addresses according to RFC5321/RFC5322)
PHONE	NANP 10-digit	(?:+1[-.\s]?)?(?\d{3})?[-.\s]?\d{3}[-.\s]?\d{4}	Accepts 555-555-5555, (555) 555-5555, +1 555 555 5555. (Regular expression to match standard 10 digit phone number)
SSN	U.S. social security	\b\d{3}-\d{2}-\d{4}\b	Rejects “000-” starts & “666”. (Add later if needed.)
CREDIT_CARD	Visa/Mastercard/AmEx	`\b(?:4\d{12}(?:\d{3})?	5[1-5]\d{14}
IP_ADDRESS	IPv4 + v6	`(?:\b\d{1,3}(?:.\d{1,3}){3}\b	(?:[A-Fa-f0-9]{1,4}:){7}[A-Fa-f0-9]{1,4})`
DOB	Dates like MM/DD/YYYY or YYYY-MM-DD	`\b(?:\d{1,2}[/-]\d{1,2}[/-]\d{2,4}	\d{4}-\d{2}-\d{2})\b`
ZIP	US ZIP / ZIP+4	\b\d{5}(?:-\d{4})?\b	Locale-specific; extend with postcodes later.

All patterns compiled with re.IGNORECASE | re.MULTILINE and wrapped in r'' raw strings.

---

2. Return-value schema

2.1 Keep the dict-of-lists for backward compatibility

from typing import Dict, List
Annotation = Dict[str, List[str]]
e.g. {"EMAIL": ["[email protected]"], "PHONE": ["555-555-5555"]}

2.2 Offer an optional structured model (new but additive)

from pydantic import BaseModel
from typing import List
class Span(BaseModel):

label: str       # "EMAIL"

start: int       # char offset

end: int         # char offset

text: str
class AnnotationResult(BaseModel):

text: str

spans: List[Span]

Why both? Existing users don’t break; new users get richer data. The regex annotator returns both:

regex_result = {lbl: [s.text for s in spans_by_label[lbl]] for lbl in spans_by_label}
return regex_result, AnnotationResult(text=input_text, spans=all_spans)

TextService will pick whichever format the caller asked for.

---

3. Performance budget

• Target ≤ 20 µs / kB on a MacBook M-series at -O.

• Compile all patterns once at module import.

• Run re.finditer for each pattern, append spans; no pandas, no multiprocessing.

---

4. Edge-case policy

• False positives > false negatives for v 1: easier to redact extra than miss PII.

• No validation (e.g., Luhn) in 4.1.0; add later under a validate=True flag.

• Reject obviously invalid IPv4 octets (25[6-9], 3\d{2}) to keep noise down.

---

5. Acceptance checklist (feeds Story 1.4 baseline)

• All patterns compile.

• Unit tests pass on happy-path and tricky strings (foo@[123.456.789.000] should fail).

• Benchmarks show regex path at least 5× faster than spaCy on 10 kB sample.

• Output dict keys exactly match label names above.