Verify LLM-extracted IOCs against a deterministic parser (close hash/CVE/MITRE trust gap)

[gadalla11111]

Summary

The LLM extraction tier merges file_hashes_*, cve_identifiers and mitre_techniques into the result dict verbatim — only wallets (_classify_wallet) and URLs (.onion split) are re-verified. A truncated hash, a hallucinated CVE-2024-99999, or an invented T9999 therefore reaches normalisation and can clear the 0.80 confidence floor on method prior alone.

Fix

New extractor/verify_gate.py applies a set-difference check: the LLM output is the required set, a deterministic parser is the present set, and required − present is rejected before merge. Each format-constrained value must pass both:

• shape — fullmatch against the strict regex already in regex_patterns (single source of truth, so the gate and the regex tier can never drift)
• grounding — the token actually appears in the source page text

Wired into extract_with_llm right after per-chunk dedup, before the merge loop. Only LLM-added, format-constrained keys are touched; regex/NER values and non-gated LLM keys pass through untouched.

Safety

Additive, zero runtime dependencies, never raises (fails open on internal error), and toggleable via VOIDACCESS_VERIFY_LLM_IOCS=false.

Testing

• 23 new tests in tests/test_verify_gate.py covering shape rejection, grounding rejection, true positives, mixed batches, passthrough, and the feature flag — all pass.
• test_extractor_llm_pipeline.py and test_llm.py stay green.
• A per-file comparison with and without this change is identical across the LLM / extractor / confidence suites (no behavioural change to existing tests).

[gadalla11111]

TL;DR — LLM-extracted file_hashes_* / cve_identifiers / mitre_techniques were merged verbatim (only wallets & URLs were re-verified), so a truncated hash or hallucinated CVE/MITRE could reach normalisation and clear the 0.80 confidence floor. This adds a shape + grounding gate that drops any LLM claim the deterministic parser can't confirm. Additive, off-switch included, no new deps.

How to test

pytest tests/test_verify_gate.py        # 23 tests: shape/grounding/passthrough/flag
VOIDACCESS_VERIFY_LLM_IOCS=false ...     # disables the gate (default: on)

The gate reuses the strict regexes already in regex_patterns, so it can't drift from the regex tier. Happy to adjust scope (e.g. gate more types, or emit verdicts to logs/metrics) if you'd prefer a different posture.