Sr. Staff Software Engineer, AI Platform

Pinterest’s AI Platform team builds tools and infrastructure for AI and ML engineers across the company. The Senior Staff Software Engineer will shape and scale a unified, reliable AI/ML evaluation platform, set technical strategy, and partner across teams to improve quality, launch decisions, and trust in AI-powered experiences.

Responsibilities:

• Identify modernization opportunities and scale the roadmap for a company-wide evaluation system in partnership with technical leadership
• Define cross-team technical strategy, align partner teams, and drive broad adoption of shared evaluation capabilities
• Make durable architectural decisions that improve the user experience and reliability of Pinterest’s AI/ML evaluation platform
• Establish evaluation methodologies and standards that balance automated metrics, expert judgment, human feedback, safety, and business outcomes
• Lead complex cross-team workstreams, resolve technical tradeoffs and risks, and deliver impact

Requirements:

• 3+ years of deep, hands-on experience designing and operating evaluation systems for GenAI and complex ML systems, including benchmark design, golden datasets, automated metrics, human evaluation, model-based grading, and failure analysis
• Strong programming skills in Python coupled with a solid grasp of distributed systems principles, and comfortable iterating with AI tools on coding and design
• Deep familiarity with GenAI and RecSys evaluation libraries and frameworks, AutoEvals, OpenEvals, Ragas, etc
• 5+ years of experience building scalable platforms for distributed experimentation, data and model versioning, orchestration, observability, release management, and production monitoring
• Strong judgment around GenAI risks, including robustness, bias, privacy, security, and the limitations of automated evaluation signals
• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent experience