Jobs via Dice is seeking a Principal Software Engineer to join their CoreAI team, which focuses on building the AI data-plane that powers LLM inferencing workloads for Microsoft and Azure customers. The role involves leading the design and implementation of core serving systems for large-scale AI services, driving innovation and ensuring high performance and reliability.
Responsibilities:
- Be a hands-on technical leader, designing, coding, and shipping core serving systems, smart routing, and request distribution for a broad portfolio of LLMs, including OpenAI, Mistral, Grok, DeepSeek, and others
- Build large-scale AI services and platform capabilities that power new products and customer experiences
- Drive cutting-edge innovation in AI systems alongside world-class engineers and cross-functional partners
- Lead through architecture, code reviews, mentorship, and technical excellence while staying close to implementation
- Improve reliability, scalability, observability, efficiency, and performance across mission-critical services
Requirements:
- Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, or Java OR equivalent experience
- Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter
- 4+ years of design and problem-solving experience, with understanding of system performance, scalability, and engineering best practices
- Understanding of distributed systems specifically in request serving at scale; (e.g. inferencing, L7 gateways, high-performance storage, distributed databases across global-scale infrastructure)
- Demonstrated experience in building high-quality, reliable systems at scale
- Experience using modern AI-assisted development tools and workflows to move faster, improve quality, and amplify engineering impact
- Customer-obsessed approach to problem solving, with empathy and a drive to deliver impactful solutions