Staff Software Engineer, Inference Performance Optimization, GenAI, DeepMind

Google DeepMind is an AI lab focused on advancing artificial intelligence through interdisciplinary research and product innovation. The Staff Software Engineer will optimize large-scale AI inference by profiling systems, identifying bottlenecks, improving throughput and latency, and developing tools and metrics to guide production capacity and cost decisions.

Responsibilities:

• Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency
• Design and implement inference optimization techniques
• Investigate and resolve complex model inference performance bottlenecks across the stack
• Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems
• Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet