Primitive Bench is focused on AI infrastructure, and they are seeking a Software Engineer to maintain and expand their open-source benchmarking platform. The role involves developing benchmarks, improving testing methodologies, and collaborating with various stakeholders to establish benchmarking standards.
Responsibilities:
- Maintaining and expanding Primitive Bench's open-source benchmarking platform for AI infrastructure primitives
- Design, implement, and improve benchmarks that provide accurate, reproducible, and vendor-neutral performance measurements across AI systems and infrastructure components
- Developing new benchmarks
- Improving existing testing methodologies
- Reviewing and evaluating benchmark results
- Maintaining developer tooling
- Ensuring the reliability and usability of the open-source repository
- Writing clean, well-tested code
- Investigating performance characteristics of AI infrastructure
- Reviewing community contributions
- Collaborating with engineers, researchers, and stakeholders to define benchmarking standards and priorities
Requirements:
- Strong software engineering fundamentals and experience building production-quality software
- Proficiency in Python
- Experience with Git and collaborative software development workflows
- Strong problem-solving skills and ability to work independently
- Excellent written and verbal communication skills
- Experience contributing to or maintaining open-source software projects
- Familiarity with AI evaluations, benchmarks, or performance measurement methodologies
- Familiarity with AI/ML infrastructure and modern developer tooling
- Experience with distributed systems, cloud infrastructure, Docker, or Kubernetes