Aditya Sankranthi

Case study

Research Agent

Plan a coverage map, gather evidence as atomic claims, audit for gaps, write the report once, then verify every citation. No orchestration framework; every design decision backed by a benchmark, not a vibe check.

43.13

GPT-5.5 RACE score on DeepResearch Bench — inside the published top-10 band

0.08 points above the published rank-9 entry (Perplexity Research, 43.05) and 1.91 above the rank-10 cutoff (Grok Deeper Search, 41.22) — on the benchmark's 50 English tasks, judged by the same GPT-5.5 RACE judge as the live leaderboard, running on claude-haiku-4-5, a small/cheap model.

04

How the score was earned

  • A coverage plan with breadth classification, so broad questions can't stop after a single supported claim.
  • Atomic evidence storage instead of a repeatedly-rewritten running summary — detail survives many search rounds instead of degrading through re-compression.
  • One write-once synthesis pass after research finishes, not a summary-of-summaries.
  • A deterministic citation-compliance postcondition: unsupported or unknown-URL citations get one bounded repair pass, then a fallback — never another unchecked model guess.

05

Architecture

  • No framework — a plain, readable Python loop, with iterative and deep research modes behind the same model and tool interfaces.
  • Provider-agnostic via one model string (litellm) — runs on hosted models or fully local through Ollama, no API key required.
  • A minimal Tool abstraction over search and page fetching.
  • Defensive and hardened: bounded retries, fail-loud, skip paths.
  • Every architecture change measured against DeepResearch Bench, an external benchmark, not graded by vibes.

06

Engineering proof

  • 127 tests passing, all network-mocked (~2s to run).
  • 50/50 English benchmark tasks completed, zero failed agent reports.
  • 2× the valid citations per report versus the pre-planner baseline (32.76 vs. 16.38), with citation validity held within 0.3 points (90.80% vs. 91.1%).
  • Dockerized single-process deploy; bring-your-own-key web UI — keys live only in localStorage, never server-side.

07 —

Stack

PythonFastAPI (SSE streaming)React + Vite + TailwindlitellmTavily / DuckDuckGoOllama

08 —

Screenshots

Ask a question
Ask a question
Researching — live search & sources
Researching — live search & sources
Cited report with automatic verification
Cited report with automatic verification
View on GitHub ↗← Back to work