Naman Vats

Naman Vats

Senior AI engineer. Agentic AI, LLM infrastructure and agent evaluation.

Based in Bengaluru · Open to relocation

I build AI agents and retrieval systems over large-scale data, investigation systems for security, and frameworks for evaluating agents at scale. My research on agent harnesses and evaluation was accepted at ICML 2026 and KDD 2026 workshops. Before moving into AI engineering, I spent years building data pipelines for 300+ enterprise clients.

Research

ICML 2026 Deep Learning for Code (DL4C) workshop
KDD 2026 Agentic AI Evaluation and Trustworthiness workshop

Around the paper

  1. Paper published.

  2. Announced on X, with 40K+ views.

    Reply from Graham Neubig, creator of OpenHands

Writing

Projects

  • Codeskills-Bench

    23 Python tasks on the code changes engineers don’t hand to AI yet: dependency migrations, merge conflicts, compliance-preserving refactors.

  • EvoSkill

    Maintainer

    Open-source framework that discovers reusable agent skills from failed runs. Cited by 60+ papers, including work from MIT, Carnegie Mellon, Microsoft, Google, Alibaba and Amazon.

  • SERA

    Retrieval and grounding infrastructure for an open-source search agent, ranked 1st among open-source agents on the DMind benchmark. 94.1% on SimpleQA, 81% on FRAMES.

  • Sentient Arena

    A program where teams submit agents against shared, unsolved benchmarks, including the OfficeQA Public Challenge.

Experience

  • Databahn.ai

    Jul 2026 – present

    Senior AI engineer

    • Founding AI engineer in Databahn’s AI Labs.
    • Agentic AI for security data pipelines.
    • Building an MCP integration system for agentic cybersecurity investigation tools.

  • Sentient Labs

    Jul 2025 – Jul 2026

    Senior software engineer, AI systems

    • Led backend development for Sentient Arena, an agent evaluation platform running benchmark tasks such as Databricks OfficeQA in Daytona sandboxes.
    • Built the retrieval and grounding infrastructure for SERA, the rank 1 open-source agent on the DMind benchmark.
    • Built a multi-hop retrieval pipeline reaching 94.1% on SimpleQA, 81% on FRAMES and 52% on SEAL, under 45 s latency.
    • Research on agent harnesses and scaffolds. Contributor to and maintainer of EvoSkill.

  • CommerceIQ

    Jul 2021 – Jun 2025

    Software engineer I to III

    • Owned data download infrastructure serving 300+ clients across 40+ daily report types from Amazon, Walmart and Instacart.
    • Led the Amazon SP-API migration, cutting failure rates by 80% and pipeline processing time by 60%.
    • Led a team of 6 to build Profit Recovery Automation, a product line generating $10–15M in annual revenue, shipped in 45 days.
    • Expanded pipelines to the Netherlands, Mexico and Poland, and cut client onboarding from 28 days to 5.

Education

Birla Institute of Technology, Mesra

Bachelor of Engineering, Electronics and Communication · 2017–2021

Contact

Email is the best way to reach me.
way2naman13@gmail.com