Member of Technical Staff — Model Optimization and Inference (Experienced)

Nuance Labs is building photorealistic, real-time AI avatars with emotional intelligence. The role focuses on making trained models fast enough for real-time use by optimizing inference across LLMs, audio models, and diffusion-based components, with attention to latency, throughput, and serving efficiency.

Responsibilities:

• Own end-to-end inference optimization across our model stack — LLMs, audio models, and diffusion-based components
• Implement and tune KV cache strategies for long-context conversations, including eviction policies, compression, and memory-efficient attention
• Evaluate, deploy, and extend inference serving frameworks (vLLM, SGLang, TensorRT-LLM, etc.) for our specific workloads
• Profile and benchmark end-to-end latency and throughput; identify and systematically eliminate bottlenecks
• Build internal tooling that makes optimization work faster and more rigorous — profiling viewers, end-to-end inference test harnesses, and other infrastructure that helps the team move quickly
• Accelerate diffusion model inference — consistency models, step distillation, caching strategies, and custom kernel optimizations
• Apply and develop quantization techniques (INT8, INT4, GPTQ, AWQ, and beyond) to reduce memory footprint and increase throughput without meaningfully degrading quality
• Work closely with research and infrastructure to ensure new models ship with optimized serving from day one