DELPHI: Build the Skeptic Into the Architecture, Not the Prompt

There is one way large language models fail at finance that no amount of prompt-tuning fixes: a confident model recalls a plausible but wrong figure, and a single-pass pipeline has nothing in it to catch the mistake. The model says a company's WACC is 8.2%, it sounds right, it flows into a DCF, and a number that was never computed becomes the basis of a recommendation. Real sell-side research solved this long before AI existed — not with smarter analysts, but with structure: specialists draft, a skeptic challenges, and compliance gates what gets published. DELPHI makes that adversarial workflow the architecture rather than a hope.

Seven agents, and a skeptic with standing

DELPHI runs a sell-side research desk as a state machine. A Research Director scopes the engagement. Four specialists — Fundamentals, Valuation, Sentiment, Macro — research in parallel against real document sets. Then the part most multi-agent systems skip: an Adversary with formal objection rights files weighted objections the specialists must rebut, with engine reruns and primary-source citations, not assertions. Finally a Compliance Auditor holds a veto over publication.

The debate is a real protocol, not a single prompt pass:

PLAN → PARALLEL_RESEARCH → ADVERSARY_ROUND_1 → REBUTTAL → ADVERSARY_ROUND_2 (max two rounds) → SYNTHESIS → AUDIT → PUBLISH or one REVISE loop

Conviction is not a vibe the model reports. It is arithmetic, recomputed every round:

conviction = base agreement across specialists − Σ(standing objection weights) − citation penalty

An objection the specialists cannot answer does not get argued away — it mechanically lowers the published conviction. That is what makes the skeptic real. A committee that always converges is one opinion wearing four hats; an Adversary whose unresolved objections cost you conviction points is a structural check.

The boundary that makes a number trustworthy

Here is the architectural decision the whole system rests on: agents make judgments; engines make calculations. The LLM never produces a number. Every figure in a published note comes from a deterministic engine that contains zero LLM code — a DCF (FCF build, CAPM WACC, Gordon and exit-multiple terminal value), comps with winsorization, a ratio suite (DuPont, Altman Z, Piotroski F, cash-conversion cycle), sensitivity grids, a tornado, and a Monte Carlo distribution. That engine carries 68 pytest tests.

Because the model cannot author a figure, a hallucinated number structurally cannot reach a published note. This is the same determinism boundary I keep coming back to: QUORUM computing every market figure in Python while six agents debate on top, RegRadar verifying every legal citation against pinned source bytes, AEOLUS letting a solver schedule and a model explain. Let the LLM reason; never let it compute.

The Compliance Auditor closes the loop — it checks every claim for a citation and every figure against the fact store before publication, and a failure loops back exactly once for a revision pass. Numbers from engines, publication behind an audit.

Honest by construction

DELPHI's data layer is keyless and cached behind a sqlite TTL store: SEC EDGAR submissions, yfinance market data, FRED macro series, StockTwits sentiment — every source free or free-tier. Six rich offline fixtures (NVDA, TSLA, MSFT, AAPL, AMZN, GOOGL) guarantee a full debate runs end-to-end with zero connectivity in about 45 seconds. The honesty shows up in the simulation default: even with no API key, every figure still comes from the real engine run and objections fire from rule triggers on the actual numbers — so NVDA argues about terminal-value dependence while TSLA argues about margin compression, because those are what the engine surfaced, not what a persona was told to say.

Coverage boundaries are labeled, not hidden. It covers publicly listed US and European names via keyless EDGAR/Yahoo sourcing; FX is converted into the quote currency (pence handled), and domicile geography is marked honestly where segment disclosure is not parsed rather than invented. The deliverable is a dual research note: machine-readable JSON plus a rendered editorial page with a football field, scenario surface, Monte Carlo distribution, global exposure map, debate transcript, and risk register.

One honest note on status: the public demo Space is being finalised, so the live link is not up yet — the full source is on GitHub and the architecture is documented in the case study below.

What this taught me

1. Build the skeptic into the architecture, not the prompt. "Consider the bear case" is a suggestion. An Adversary with objection rights whose unresolved points cost you conviction is a control.
2. The boundary is the product. Once the LLM physically cannot author a number, the entire class of confident-wrong-figure failures disappears — and you stop having to trust the model on the one thing it is worst at.
3. Conviction should be earned arithmetic. A score that is computed from agreement minus standing objections minus citation penalties is auditable. A score the model just announces is theater.

Equity research is the hardest possible audience for "the AI made up a number," which is exactly why it is the right place to prove the pattern. DELPHI is that proof: a desk where what gets published has survived an argument and a numeric audit, on keyless public data.

Source on GitHub → · Full case study →