Staff/ Principal Software Engineer (L6/L7): AI Platform Model Serving

Netflix is rebuilding its AI/ML model-serving platform to support real-time, asynchronous, and agentic workloads across in-house infrastructure, third-party hosting, and external LLM providers. The Staff/Principal Software Engineer will set technical direction for this platform, improve inference efficiency and reliability, and support AI capabilities across member-facing products and internal tools.

Responsibilities:

• You will tackle the efficiency problems that most directly drive cost per inference and capacity: disaggregated serving, intelligent routing across models and providers, speculative decoding, prompt caching, and multi-GPU inference
• You will lead the technology behind some of Netflix’s largest bets, embedding AI capabilities into member-facing products at global scale
• You will also build durable, resumable execution for long-running agentic workloads
• This is a rare very senior level opportunity to set and own the technical direction for a platform at the intersection of massive production scale, aggressive cost/ utilization engineering, and genuinely unsolved problems in GenAI serving by leading the technical architecture and directions for a growing engineering organization

Requirements:

• Have a BS/MS/PhD in Computer Science, Applied Math, Engineering, or a related field
• With relevant experience in the AI/ ML Infrastructure and distributed systems for 12+ years
• Influence without authority to guide architectural decisions and alignment across multi-teams through trust, data, and demonstrated expertise
• Shape long-term technical roadmaps (typically looking 2 to 3 years ahead) and ensure scalability, maintainability, and business alignment
• Work closely with product managers, operations, and executive stakeholders to balance technical trade-offs with business priorities
• Clear Communication skill to translate dense technical concepts for non-technical stakeholders and produce clear documentation, architecture decision records (ADRs), and proposals
• Have experience with high-traffic distributed services and infrastructure for AI/ ML model inference, with a focus on high availability, security, and performance
• Have experience building scalable model-serving solutions for Foundation Models that powers generative and predictive AI applications with practical skills in reducing latency and costs
• Familiar with deploying AI/ ML models using platform and solutions such as NVIDIA Triton Inference Server, vLLM, or SGLang, and scaling them via Kubernetes
• Have experience working with public cloud infrastructure providers such as AWS, Azure, or GCP
• Have open-source contributions to projects in the AI/ML, distributed systems, or infrastructure space
• Hands-on experience with building AI models, and have explored optimization techniques for AI models, such as quantization, model compression, or model pruning
• Proficient in leveraging AI coding tools in software development
• Have experience working with model gateway and intelligent routing for choosing the right AI models based on context of the requests
• Exposure to deploying multi-adapters model serving, disaggregated prefill/decode, or large-scale KV cache management
• Working with evaluation and benchmarking tools for LLMs and AI Agents
• Knowledge of GPU topology, NVIDIA CUDA programming, and machine learning compilers