Engineering Manager, Machine Learning, Model Evaluations and Data Curation (AI Foundations)

Netflix is a leading entertainment company focused on pushing the boundaries of storytelling and technology. They are seeking an Engineering Manager to lead a team responsible for model evaluations and data curation for large language models, ensuring that these models improve personalization and discovery for users.

Responsibilities:

• Partner with downstream AI application teams to define shared evaluations that codify application expectations of LLMs and other foundation models, ensuring progress can be transparently tracked against real-world needs
• Design rigorous benchmarks and evaluation methodologies across ranking & recommendations, content understanding, and language/text generation — grounded in a deep technical understanding of LLMs, their strengths, limitations, and failure modes
• Lead the development of evaluators and strong baselines to ensure in-house LLMs and other foundation models demonstrate clear advantages over off-the-shelf alternatives
• Build scalable, reproducible data and evaluation systems that make dataset creation and evaluation design as nimble and experiment-friendly as model development itself
• Hire, grow, and nurture a world-class team, fostering an inclusive, high-performing culture that balances research innovation with engineering excellence
• Work closely with the teams developing Netflix’s foundation models (including our core LLM) to ensure evaluation and data insights are folded back into the cadence of model development. Proactively influence the ML Platform and Data Engineering teams at key interfaces

Requirements:

• Experience building and leading high-performing teams of ML researchers and engineers
• Proven track record of leading machine learning initiatives from research to production, ideally involving evaluation frameworks, ML infrastructure, or data-intensive systems
• Strong technical expertise in LLMs, their evaluation, and practical methods for ensuring robustness, reproducibility, and quality
• Broad knowledge of machine learning fundamentals and evaluation methodologies, including benchmark design, model-based evaluators, and offline/online metrics
• Experience driving cross-functional projects, including close collaboration with AI application teams to translate product needs into evaluation frameworks
• Excellent written and verbal communication skills, able to bridge technical and non-technical audiences
• Advanced degree in Computer Science, Statistics, or a related quantitative field
• 8+ years of overall experience, including 3+ years in engineering management
• Experience with large-scale ML systems and foundation models, especially LLMs
• Background in building evaluation frameworks, model benchmarking, or data infrastructure for LLM training
• Familiarity with multi-modal data and evaluation