Senior Software Engineer, ML Infrastructure

SambaNova is an inference-first company focused on generative AI solutions. The Senior Software Engineer, ML Infrastructure will design, build, and operate production-grade inference infrastructure, ensuring high-performance and reliable services for customers.

Responsibilities:

• Design and productionize advanced inference techniques on RDU to optimize for performance and cost. Key areas include speculative decoding, constrained decoding, function/tool calling, prompt caching, and long-context inference
• Own SambaNova's integration with vLLM and adjacent serving frameworks, adapting them to RDU's architecture
• Own the public inference API surface exposed through SambaStack and SambaCloud
• Build and maintain the accuracy verification and regression infrastructure that gates every inference feature shipped to customers
• Partner with ML, compiler, runtime, and product teams to take inference features from prototype to production
• Contribute to technical design discussions, code reviews, and architectural decisions as a senior individual contributor

Requirements:

• Bachelor's degree in Computer Science, Electrical Engineering, or related field
• 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving
• Strong software engineering fundamentals: algorithms, data structures, concurrency, and systems design
• Experience designing and maintaining production services with strict latency, throughput, and availability requirements
• Working knowledge of modern LLM inference techniques and familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang
• Proficiency in Python
• Experience collaborating across teams to deliver complex, system-level engineering solutions