Software Engineer, Agent Optimization & Evaluation

Reddit is building a next-generation AI developer platform to help engineers build, test, review, and ship software efficiently. The Software Engineer, Agent Optimization & Evaluation will lead the platform for evaluating and improving AI agents, including evaluation environments, model orchestration, and production feedback loops.

Responsibilities:

• Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows
• Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks
• Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback
• Design model routing and orchestration strategies that optimize for quality, latency, and cost
• Define metrics and experimentation frameworks that measure developer productivity and engineering velocity
• Build tooling to analyze agent behavior, investigate failures, and accelerate iteration
• Partner closely with Platform Engineering, Infrastructure, and Developer Productivity teams to integrate agent capabilities into engineering workflows
• Help shape Reddit's long-term strategy for a flexible, model-agnostic agent platform that gives us control over our AI capabilities and economics

Requirements:

• 5-12+ years of engineering experience
• Extensive experience building and shipping large-scale production software systems
• Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems
• Strong understanding of modern LLMs, AI agents, and the engineering systems required to deploy them effectively
• Experience building systems that optimize across multiple dimensions, including quality, latency, reliability, and cost
• Excellent software engineering fundamentals and a passion for building production-quality systems
• Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives
• Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations