Senior/Staff Applied AI Engineer

Tali AI is building an AI-powered clinical operating system that helps clinicians reduce paperwork and improve healthcare accessibility. The Senior/Staff Applied AI Engineer will own AI systems end to end, including models, prompts, retrieval, data, evaluations, agents, and production debugging. The role focuses on building reliable evaluation pipelines, diagnosing failures, improving AI systems, and setting applied AI engineering standards.

Responsibilities:

• Build the evaluation pipelines that decide whether the AI is good enough to ship. Automated judges, regression suites, human review, and the datasets underneath them. On the audio path that means word and speaker error rates and the audio-quality measures that tell you a recording was worth trusting
• Diagnose failure modes and fix them. The lever might be the prompt, the retrieval, the routing, the model, the audio capture, or a fine-tune
• Build production agents. Tool use, orchestration, guardrails, and recovery when a step fails. You also build what sits under them: search, vector storage, and the harness the agents run in
• Debug one visit end to end, then find every case like it. You trace a single interaction from audio to delivered note and work out what broke. Then you slice the warehouse to size the problem and prove the fix
• Decide which model serves which request, and change that safely. Weighted routing, staged rollout, and attribution good enough that you know which change moved the number
• Own these systems in production. You get the alert when quality slips, you find the cause, and you decide what ships to fix it
• Turn a vague clinical complaint into a problem statement, a metric, and a plan the team can act on
• Set the bar for how Tali does applied AI. Your evals become the evals everyone else runs