Fraud Triage Agent
A fraud investigator's co-pilot for SEPA-native, BaFin-regulated banking
Situation
German neobank N26 was fined €4.25M (2021) and €9.2M (2024) by BaFin, Germany's financial regulator, specifically for systematic delays in filing suspicious activity reports. A 2024 special audit found serious deficiencies in risk management and complaint handling, leading to a second special monitor being appointed. The root cause across every enforcement action: compliance infrastructure that couldn't scale with transaction volume. The bottleneck wasn't fraud detection — N26 already had an upstream fraud-scoring model. It was the reasoning step between a flag and a human decision: investigators manually reviewing 50–200 flagged transactions per shift with no structured context, no explainability, and no BaFin-auditable decision log — taking 10–20 minutes on novel or conflicting-signal cases.
Task
Design and build a decision-support prototype — inspired by public N26 job postings and BaFin's public enforcement findings, using only public data — that would let a fraud investigator go from flagged transaction to a defensible, logged decision in under 3 minutes, while satisfying EU AI Act Article 14 human-oversight requirements for a system that would be classified high-risk under Annex III. Explicit constraint I set for myself: the system must never auto-act. Given the regulatory context (BaFin fines for inadequate oversight, EU AI Act Article 14), a "smarter" agent that skips human review would fail the actual job to be done — this had to be judgment-augmentation, not automation.
Action
I scoped, built, and evaluated this in 14 days: 1. User research framing: defined the user (N26 Financial Crime investigator) and job-to-be-done (context-gather → decide → log a defensible rationale) before writing any code, and identified that steps 1–3 of that job were entirely manual today. 2. Architecture decision: chose a LangGraph state graph over a single-shot LLM chain — five explicit nodes (retrieve case → analyze in pandas → LLM reasoning → HITL escalation gate → SQLite audit log) — so the human-in-the-loop boundary is enforced in code, not left as a UI convention. 3. Made a deliberate build trade-off: pushed velocity counts, threshold checks, and amount comparisons into deterministic Python/pandas rather than asking the LLM to do arithmetic over tabular data — because LLMs hallucinate on that, and structured findings are what the model should reason over, not compute. 4. Designed for regulatory reality, not just UX: built AML/SAR routing as a first-class output distinct from fraud handling (German banks legally can't tell a customer about an AML suspicion — it goes to the FIU, not customer contact), and mapped every design decision to EU AI Act Art. 14, GDPR Art. 22, PSD2, and GwG/AML requirements in a compliance table. 5. Built an evaluation harness, not just a demo: tested against a 50-case held-out set from the public IEEE-CIS dataset, tracking agreement rate, calibration (fraud cases wrongly approved — the metric that actually matters under BaFin/PSD2 liability), and escalation rate — and documented reasoning-quality limitations transparently rather than hiding them. 6. Scoped ruthlessly: explicitly cut and documented five things (AML routing UI, recalibration dashboard, case queue view, calibration plot vs. table, dataset homogeneity limitation) as a 14-day prototype rather than pretending it was production-complete.
Result
Result A working prototype (LangGraph + Claude Sonnet 4.6 + Streamlit) deployed live, with full source on GitHub — not a slide deck. 100% agreement rate and 0% fraud-cases-incorrectly-approved on the held-out eval set, with a 50% escalation rate by design (reflecting EU AI Act Article 14 compliance, not a tuning failure). A complete, mapped-out regulatory compliance framework (EU AI Act, BaFin/MaRisk, GDPR, PSD2, GwG) built into the architecture from day one — demonstrating the ability to design for high-risk AI classification rather than bolting compliance on afterward. Defined the 90-day success metrics (review time, override rate, SAR latency, calibration accuracy) I'd use to argue for continued investment if this were a real pilot — showing product thinking beyond "ship and walk away." Transparently documented what was cut and why, and named a real limitation (dataset homogeneity from the IEEE-CIS transformation) — signaling rigor and honesty over a polished-looking demo.
Closing Notes
This project isn't a claim that I can replace a fraud-ops team's judgment — it's a demonstration of how I think about AI product work: start from a documented regulatory failure, define the actual job-to-be-done, put the human decision at the center of the architecture rather than the edge of it, and evaluate honestly rather than optimistically. If you're hiring for AI Product Management or AI Consulting in a regulated European market, I'd welcome the chance to walk through the reasoning behind any of these decisions.