AI Accelerator Software Principal Engineer- Framework Integration

Ampere is a semiconductor design company focused on high-performance, energy efficient AI compute. The AI Accelerator Software Principal Engineer – Framework Integration will lead the design and delivery of high-performance deep learning inference solutions, optimizing popular frameworks to run efficiently across various environments.

Responsibilities:

• Integrate and optimize deep learning frameworks—such as PyTorch, ONNX, and llama.cpp—for the Ampere deep learning accelerator backend, enabling efficient and correct execution across a wide set of model types
• Go deep into the full software/hardware execution stack, including: inference serving and orchestration, framework integration layers, compiler and graph/runtime support, runtime libraries and user-mode execution paths, compute kernel development, profiling, benchmarking, and performance tuning
• Improve both performance and accuracy for models using popular frameworks, and ensure compatibility with serving ecosystems such as vLLM and SGLang—helping deliver production-ready inference behavior
• Partner with hardware and platform teams to co-optimize AI execution for better outcomes: increased throughput, reduced latency, improved scalability, better resource utilization (compute/memory/IO), higher sustained performance under realistic workloads
• Contribute to the development of software and hardware AI co-processors/accelerators, delivering reusable libraries, optimized execution paths, and robust integration with existing tooling
• Work closely with cross-functional teams (compiler/runtime, kernels, platform, and product engineering) to integrate AI capabilities into Ampere’s cloud-native processor platforms and accelerators