Senior Engineer, Applied AI

Ro is a vertically integrated healthcare technology company connecting telehealth, diagnostics, pharmacy, and logistics to deliver patient care. Ro is seeking a Senior Engineer, Applied AI to build and ship LLM-powered features for patient experience, clinical operations, and internal tooling, owning AI systems from prototype through production. The role focuses on orchestration, retrieval, evaluations, observability, safety, reliability, and collaboration with clinical operators.

Responsibilities:

• Design, build, and ship production LLM powered features end-to-end
• Build the AI application layer, including prompt orchestration, tool calling, retrieval (RAG), embeddings, agent workflows, and structured outputs
• Develop evaluation infrastructure, including regression suites, LLM-as-a-judge evaluations, synthetic datasets, human review workflows, and quality metrics that catch regressions before users do
• Design observability systems that measure accuracy, latency, cost, hallucination rates, and model behavior in production
• Build safety and reliability guardrails that ensure AI systems meet quality, compliance, and operational requirements
• Evaluate and integrate third-party tools, including orchestration frameworks, evaluation platforms, vector databases, and model providers; and make pragmatic build-versus-buy decisions
• Raise engineering standards across the team through technical leadership, code reviews, mentorship, and thoughtful system design
• Sit with EHR operators, Patient Advocates, and clinical teams to understand the workflows you're automating
• Prototype quickly, ship early, measure outcomes, and iterate based on real-world usage