Senior Engineer II, AI Inference Engine Systems

DigitalOcean is a cutting-edge technology company focused on simplifying cloud and AI solutions. They are seeking a Senior Engineer II to lead the design and development of high-scale AI inference systems, ensuring optimal performance and reliability for their services.

Responsibilities:

• Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models
• Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards
• Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing
• Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs
• Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement
• Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health