Case study
Research Agent
An agent that shows its work: plan a coverage map, gather evidence as atomic claims, audit for gaps, write the report once, then mechanically verify every citation. No orchestration framework; every design decision backed by a benchmark, not a vibe check.
Citation accuracy on the FACT metric, the highest score on DeepResearch Bench's public leaderboard
Outperforms dedicated deep-research products from Anthropic, OpenAI, Google, and Perplexity on citation accuracy, running on a small, low-cost model (claude-haiku-4-5). The same system scores 43.13 on the RACE metric, which lands inside the benchmark's published top-10 band: 0.08 points above the rank-9 entry (Perplexity Research, 43.05).
Stack
AI & LLM
Retrieval & grounding
Evaluation
Backend
Frontend
Dev & deploy
How the score was earned
- A coverage plan with breadth classification, so broad questions can't stop after a single supported claim.
- Atomic evidence storage instead of a repeatedly-rewritten running summary, so detail survives many search rounds instead of degrading through re-compression.
- One write-once synthesis pass after research finishes, not a summary-of-summaries.
- A deterministic citation-compliance postcondition: unsupported or unknown-URL citations get one bounded repair pass and then a fallback, never another unchecked model guess.
Architecture
- No framework: a plain, readable Python loop, with iterative and deep research modes behind the same model and tool interfaces.
- Provider-agnostic via one model string (litellm), so it runs on hosted models or fully local through Ollama, no API key required.
- A minimal Tool abstraction over search and page fetching.
- Defensive and hardened: bounded retries, fail-loud, skip paths.
- Every architecture change measured against DeepResearch Bench, an external benchmark, not graded by vibes.
Engineering proof
- 127 tests passing, all network-mocked (~2s to run).
- 50/50 English benchmark tasks completed, zero failed agent reports.
- 2× the valid citations per report versus the pre-planner baseline (32.76 vs. 16.38), with citation validity held within 0.3 points (90.80% vs. 91.1%).
- Dockerized single-process deploy, plus a bring-your-own-key web UI where keys live only in localStorage, never server-side.
Screenshots
