Case study
Research Agent
Plan a coverage map, gather evidence as atomic claims, audit for gaps, write the report once, then verify every citation. No orchestration framework; every design decision backed by a benchmark, not a vibe check.
43.13
GPT-5.5 RACE score on DeepResearch Bench — inside the published top-10 band
0.08 points above the published rank-9 entry (Perplexity Research, 43.05) and 1.91 above the rank-10 cutoff (Grok Deeper Search, 41.22) — on the benchmark's 50 English tasks, judged by the same GPT-5.5 RACE judge as the live leaderboard, running on claude-haiku-4-5, a small/cheap model.
04 —
How the score was earned
- A coverage plan with breadth classification, so broad questions can't stop after a single supported claim.
- Atomic evidence storage instead of a repeatedly-rewritten running summary — detail survives many search rounds instead of degrading through re-compression.
- One write-once synthesis pass after research finishes, not a summary-of-summaries.
- A deterministic citation-compliance postcondition: unsupported or unknown-URL citations get one bounded repair pass, then a fallback — never another unchecked model guess.
05 —
Architecture
- No framework — a plain, readable Python loop, with iterative and deep research modes behind the same model and tool interfaces.
- Provider-agnostic via one model string (litellm) — runs on hosted models or fully local through Ollama, no API key required.
- A minimal Tool abstraction over search and page fetching.
- Defensive and hardened: bounded retries, fail-loud, skip paths.
- Every architecture change measured against DeepResearch Bench, an external benchmark, not graded by vibes.
06 —
Engineering proof
- 127 tests passing, all network-mocked (~2s to run).
- 50/50 English benchmark tasks completed, zero failed agent reports.
- 2× the valid citations per report versus the pre-planner baseline (32.76 vs. 16.38), with citation validity held within 0.3 points (90.80% vs. 91.1%).
- Dockerized single-process deploy; bring-your-own-key web UI — keys live only in localStorage, never server-side.
07 —
Stack
PythonFastAPI (SSE streaming)React + Vite + TailwindlitellmTavily / DuckDuckGoOllama
08 —
Screenshots


