Sr. Backend Software Engineer (Infrastructure ML Ops)

Philo is hiring a Senior Backend Software Engineer to build and improve infrastructure for ML and LLM workloads. The role focuses on model serving, evaluation, cloud and Kubernetes infrastructure, observability, and tools that support engineering teams across the company.

Responsibilities:

• Manage the core infrastructure at Philo with a focus on supporting ML and LLM workloads. This infrastructure is relied on by a large subscriber base of Philo’s product as well as internal stakeholders across the company
• Design, build, and maintain cloud infrastructure components (AWS and Kubernetes) using a combination of in-house technology and open source software
• Build and scale model serving and inference infrastructure for both real-time recommendation models and LLM workloads — including model servers (e.g. NVIDIA Triton), LLM inference engines (e.g. vLLM, TGI, TensorRT-LLM), autoscaling, and latency/throughput optimization
• Manage GPU and accelerator infrastructure on Kubernetes — scheduling, capacity and quota management, and cost optimization across on-demand and spot capacity
• Design, build, and maintain monitoring and observability systems — spanning both system health and model quality (drift, evaluation, and online performance) — that enable Philo engineers and the support team to gain insight and to discover and debug issues
• Build and operate model evaluation infrastructure: offline and online eval pipelines, golden/regression datasets, LLM-as-judge harnesses, A/B testing and experimentation, and feedback/ground-truth loops
• Design, build, and maintain CI/CD systems enabling Philo engineers to create pipelines to test and deploy their code
• Establish tools, methods and best practices for other Philo engineers interfacing with the Philo ML infrastructure. Ensure reliability, security, and scalability of the Philo platform. Promote Infrastructure as Code
• Work closely with other Philo engineers to deploy and instrument software systems
• Drive evaluation, selection, and integration of third-party vendor systems and work closely with vendors to configure and manage them