Agentic RAG benchmark for heterogeneous technical corpora
Built an end-to-end evaluation system over PDFs, HTML, DOCX, CSV and tabular data to test where retrieval architecture and agentic routing actually improve answer quality — and where they only add latency.
Factual, multi-hop, cross-document, tabular, and unanswerable cases.
Recall@10 improvement across controlled retrieval experiments.
NDCG@10 improvement using hybrid retrieval and reranking.
Lower latency for simpler RAG on workloads where agent routing added no quality.
What mattered
Separating retrieval quality from generation quality. Treating latency and token cost as first-class evaluation dimensions. Designing tests that can expose when “more agentic” is not “better.”
Python · BM25 · dense embeddings · RRF · cross-encoders · vLLM · llama.cpp · RAG evals