AI/LLM Safety Engineer

Propio is hiring an AI/LLM Safety Engineer to ensure its models and agents behave safely in production, with a primary focus on agent safety, Trust & Safety, and Responsible AI. The role designs safety evaluations and guardrails, leads red-teaming and threat modeling, monitors production systems, and supports governance reviews.

Responsibilities:

• The AI/LLM Safety Engineer will join our AI team and take ownership of how safely our models and agents behave in production; with a focus on AI Safety, Trust & Safety, and Responsible AI
• You will design the evaluations that catch unsafe behavior, build the guardrails that stop it, and lead the red-teaming that finds the gaps before our users—or attackers—do
• Agent safety is the primary focus of this role: you will help ensure that as our systems gain the ability to call tools and take actions, they do so within well-defined, well-tested boundaries
• Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships
• Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure
• Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting
• Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users
• Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls
• Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves
• Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production
• Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards
• Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams