Principal Software Engineer- AI Frameworks

Microsoft’s AI Frameworks team develops software, performance systems, and engineering tools that enable state-of-the-art AI models to run reliably and efficiently at cloud scale. The Principal Software Engineer will set technical direction and lead high-impact initiatives across AI frameworks, performance engineering, benchmarking, and tooling while delivering hands-on solutions that improve model velocity, platform efficiency, and production reliability.

Responsibilities:

• Define technical vision, architecture, and multi-release strategy for critical AI framework, performance, benchmarking, or developer-productivity capabilities
• Lead ambiguous, cross-stack investigations and investments spanning models, frameworks, compilers, runtimes, systems, services, and silicon
• Establish common measurement, automation, observability, and engineering mechanisms that turn one-off analyses into scalable platform capabilities
• Drive measurable improvements in model onboarding velocity, runtime performance, reliability, hardware utilization, and Azure capacity efficiency
• Influence architecture and priorities across teams; align researchers, product groups, infrastructure owners, and hardware partners around clear decisions and execution plans
• Provide hands-on technical leadership through prototypes, critical-path implementation, design and code reviews, complex debugging, and operational readiness
• Raise the engineering bar by mentoring engineers, developing technical leaders, and advancing standards for quality, maintainability, and inclusive collaboration
• Embody Microsoft’s culture and values

Requirements:

• Bachelor's Degree in Computer Science or a related technical field and 6+ years of technical engineering experience coding in languages such as C++, or Python, or equivalent experience
• Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
• Deep expertise in GPU or equivalent accelerator programming, compilation, and low-level execution, including intermediate representations, lowering, code generation, instruction-level behavior, memory hierarchy, and synchronization
• Expertise in parallelism strategies used in LLM training and inference, including tensor, pipeline, data, and expert parallelism, with the ability to evaluate their suitability for different models and hardware configurations
• Expertise in distributed inference acceleration, including prefill/decode disaggregation, KV-cache transfer, collective communication, and compute/communication overlap
• Strong understanding of LLM serving architectures such as vLLM, SGLang, or equivalent systems, with a track record of translating architectural improvements into measurable production gains
• Demonstrated leadership of cross-team technical initiatives from strategy and design through implementation, deployment, and sustained production impact
• A track record of creating reusable platforms, influencing stakeholders, and mentoring engineers
• Ability to use AI-assisted development tools effectively and establish practices that improve engineering productivity without compromising correctness, performance, or maintainability