Skip to content

Research note

AEGIS Live: Real-Time AML Where Latency Is a Feature and Train/Serve Skew Is the Enemy

Money laundering is confirmed retrospectively — live blockchain data never arrives with a "laundered: yes/no" label. So you score in real time and humans confirm later. AEGIS is the full streaming stack: resilient ingestion, a backpressure bus, a sliding-window graph, a graph/ML ensemble, single-digit-ms p95, and a parity gate that kills train/serve skew.

Siddharth Jain4 min readStreamingAMLMachine LearningData EngineeringFinTech

Here is the honest frame most "AI catches money laundering" demos quietly skip: live blockchain data never arrives with a "laundered: yes/no" label. Laundering is confirmed retrospectively, by investigators, months later. So a credible system does exactly what every real production AML stack does — it scores in real time, and humans confirm later. Live red flags are predictions; only sanctions exact-hits are ground-truth designations. AEGIS Live is the full streaming stack built around that honest frame, shipped solo, on free infrastructure, for $0.

Try the live demo →

A live wall, scoring the instant a transaction arrives

AEGIS is a surveillance wall of real cryptocurrency transactions streaming onto the screen, each scored for money-laundering risk the moment it lands. The ticker moves, the throughput counter climbs, the network graph grows, and when a transaction touches the dirty corners of the network it blooms red — with the reason and the subgraph behind it, plus an auto-drafted Suspicious Activity Report.

The whole 14-part blueprint is built, end to end:

resilient WebSocket ingestion → event bus with backpressure → sliding-window graph → live feature builder → sanctions + ML + anomaly ensemble → alert engine → subgraph explanation + drafted SAR → WebSocket hub → the Wall

It ingests the real Bitcoin mempool live (a Coinbase/exchange feed drives the price ticker and USD values). Ingestion is resilient by design — per-feed reconnect, exponential backoff, heartbeat — and the event bus is a bounded queue with drop-oldest backpressure and adaptive sampling when it backs up, so a burst degrades gracefully instead of blowing up memory. A sliding-window in-memory NetworkX graph ages nodes out so the structure stays current.

Latency is a feature here

Most ML systems treat latency as an afterthought. In live surveillance it is a product requirement. The fast path — a LightGBM model plus a set-lookup sanctions screen — is engineered to a single-digit-millisecond p95, measured (p50/p95/p99 are displayed) and enforced by a CI gate. Heavier graph work runs off the hot loop. The ensemble fuses three signals — a sanctions screen (exact plus N-hop), the LightGBM model, and an IsolationForest structural-anomaly detector — into one score with a human-readable reason; the alert engine adds threshold, dedup, and rate-limiting so the wall flags signal, not noise.

The silent killer: train/serve skew

If I had to name the single thing that quietly destroys streaming ML in production, it is train/serve skew — the features you compute live drifting subtly from the features the model trained on, so the model is scoring inputs it never really saw. AEGIS kills it structurally: the live feature builder and the trained model share one FEATURE_NAMES list, and a parity CI gate fails the build on any drift. This is the streaming-data cousin of the leakage discipline at the heart of CreditForge — in both, the unglamorous correctness work (point-in-time targets there, feature parity here) is what separates a model that works in a notebook from one that works on data it has not seen.

Train/serve skew is the silent killer of streaming ML. One shared feature list, enforced by a gate, is the fix.

Honest labels, graceful degradation

The labelling is deliberately honest: only sanctions exact-hits are marked "confirmed"; everything else is "risk / suspected," because live data has no laundering label and claiming otherwise would be a lie. Detection quality is measured as PR-AUC against synthetic AMLSim-style typologies — fan-in/out, peeling chains, pass-through — computed in live-parity features and swappable for the labeled Elliptic dataset when you want history.

And because a live wall must always have motion, every layer degrades instead of dying: feed down falls back to a synthetic source after an 8-second grace, a missing model falls back to a heuristic, the LLM down falls back to a deterministic template SAR, Postgres unset falls back to SQLite. A nightly GitHub Action refreshes the OFAC sanctions list, and aegis/eval/gates.py runs in CI on every push — feature parity, fast-path p95, and detection PR-AUC, all gated.

What this taught me

  1. State the honest frame first. "Live data has no laundering label, so we predict in real time and humans confirm later" is more credible than any accuracy number, and it shapes the whole design.
  2. Latency is a feature you engineer, not an accident you measure. A single-digit-ms p95 budget, displayed and CI-gated, is a product decision.
  3. Feature parity is the streaming version of leakage discipline. One shared FEATURE_NAMES list and a parity gate is the difference between a model that holds up live and one that silently rots.

Real-time AML is the rare problem that demands the entire stack at once — resilient ingestion, backpressure, a streaming graph, a graph/ML ensemble, sanctions screening, and a GenAI SAR layer — under a hard latency budget. Building it solo, for $0, was the point: this is what the full data-engineering-plus-ML-plus-GenAI stack looks like when one person has to make every layer hold.

Try the live demo → · Source on GitHub → · Full case study →

Request coverage

I am pivoting from manufacturing AI to finance: open to roles, mentorship, and collaborators in fintech, quant, and bank AI.