Sr. Principal Software Engineer

Cerence AI is the global leader in AI for transportation, specializing in building AI and voice-powered companions for vehicles. They are seeking an exceptional Senior Principal Software Engineer to optimize and deploy high-performance inference pipelines and drive improvements in model performance and deployment efficiency.

Responsibilities:

• Optimize and deploy high-performance LLM inference pipelines
• Own inference runtimes across data center, edge, and embedded platforms
• Push model performance through quantization, kernel fusion, and cache optimization
• Drive latency and throughput improvements that directly impact production products
• Enable efficient, reliable deployment without external vendor dependency
• Build deep expertise and ownership of: vLLM, TensorRT-LLM, llama.cpp, QAIRT
• Extend and tune inference engines using custom CUDA kernels
• Adapt runtimes for constrained and embedded deployment environments
• Implement and evaluate quantisation strategies: INT8, INT4, FP4, FP8, mixed precision, AWQ, GPTQ
• Balance accuracy, latency, memory footprint, and throughput
• Optimize key–value cache performance through: Paging, Prefix caching, Cache-aware memory layout design
• Reduce memory pressure while sustaining high throughput
• Design and tune: Batching strategies, Continuous batching, Speculative decoding
• Optimize tail latency and tokens/sec under real production traffic patterns

Requirements:

• Proven experience optimizing ML inference performance in production
• Deep understanding of GPU architecture and memory hierarchies
• Hands-on experience with CUDA and low-level performance tuning
• Experience deploying models beyond research environments
• Inference engines: vLLM, TensorRT-LLM, llama.cpp, QAIRT
• CUDA kernel development and profiling
• Quantisation techniques: INT8/INT4/FP4/FP8, AWQ, GPTQ
• KV cache optimisation and memory layout design
• Latency optimisation: batching, speculative decoding, continuous batching