Coverage note · Regulation & Compliance · RegTech · Agentic AI
RegRadar — EU Regulatory-Impact Engine
An agentic engine that turns the EU regulatory firehose into a verified, ranked, bank-ready action list — without hallucinating a single citation.
Key metrics
1. F1 0.957 — live extraction graded vs a hand-labeled DORA oracle (Reg (EU) 2022/2554, 64 EUR-Lex articles); precision 0.917 · recall 1.000; 100% citation integrity on accepted obligations.
The problem
The EU ships regulation faster than any compliance team can read it — DORA, MiCA, the AI Act, CRR3, NIS2, each one dozens to hundreds of articles of concrete obligations a bank must translate into systems, controls, and deadlines. Today that translation is manual, slow, and expensive.
The obvious fix — point an LLM at the text — fails in exactly the way a regulated industry can least afford: a fluent, confident claim citing an article that does not say what the model says it does. A citation that does not hold up is not a bug here, it is a liability.
Architecture
- 01
Source-Monitor
Watches the EUR-Lex / CELLAR firehose for new and amended acts.
- 02
Bronze store
Raw act pinned by content hash — immutable and idempotent; re-ingesting identical bytes is a no-op.
- 03
Parser → Silver
Formex / EUR-Lex XHTML parsed into a structured article tree.
- 04
Obligation extractor
LLM pulls the concrete "you must…" duties out of each article.
- 05
Citation verifier
Deterministic: rejects any obligation whose anchor quote is not found verbatim in the pinned source; opens a human gate.
- 06
Impact mapping
Maps surviving obligations to the bank's controls; gaps surface in red.
- 07
Prioritization
Ranks the gaps by deadline, effort, and risk.
- 08
Memo + human gate
Drafts the gap-assessment memo in EN + DE, approve/reject, DOCX/HTML export.
Key tradeoffs
Determinism boundary — LLM extracts and narrates only; all matching, hashing, and scoring is pure Python.
WhyA model that returns different obligations on Tuesday than Monday is not deployable in compliance. The math has to be reproducible and auditable.
Citation integrity by construction — verify the quote against pinned bytes.
Why"Please only cite real sources" is a prompt, not a control. A verifier that checks the anchor against the immutable source is.
Provider failover chain: Groq → Gemini → OpenRouter → Ollama → deterministic floor.
WhyNo single point of quota failure. When a free tier caps out, the system degrades and opens a human gate — it does not crash.
Article-capped extraction + exact response cache.
WhyKeeps the whole thing inside a $0 free-tier token budget; already-extracted articles never re-spend tokens.
Eval results
Live extraction graded vs a hand-labeled DORA oracle (Reg (EU) 2022/2554, 64 EUR-Lex articles), Groq→Gemini failover.
Accepted obligations that matched the oracle.
Oracle obligations the system recovered.
Share of cited authorities found verbatim in the pinned source. The verifier rejected 2 ungrounded anchors over-extracted from Article 19 and opened a human gate.
Production proof
The artifact that keeps the numbers honest: the eval harness and monitoring gates that run in CI, not a one-off notebook result.
Eval-as-CI-gate
CI · PASSINGregradar/eval/run.py runs as a CI gate and exits non-zero on regression — a model or prompt change that drops F1 or citation integrity fails the build.
Turns a multi-day manual gap assessment into a verified, ranked, bilingual memo in minutes — the leverage a bank's compliance function pays for, delivered with the auditability a regulator demands.
Request coverage
I am focused on finance AI: credit risk, RegTech, AML, and agentic investment research. Open to roles, mentorship, and collaborators in fintech, quant, and bank AI.
Related notes