Senior Software Engineer, AI Developer Experience

Vanta is hiring a Senior Software Engineer for its AI Developer Experience team. The role builds and improves AI-enabled engineering infrastructure, agent evaluation systems, and developer workflows to make AI tools reliable and useful for Vanta engineers.

Responsibilities:

• Plan, build, and improve background-agent infrastructure that enables reliable, trackable, and repeatable automated engineering work
• Develop systems for evaluating agent performance using task outcomes, quality measures, human intervention, cost, and other signals
• Create evaluation sets and feedback loops that help AI systems improve over time through regular review, tuning, and correction
• Improve how engineers discover, understand, and use shared AI tools and workflows
• Evaluate AI development tools, including code generation, AI-assisted review, and testing, against real engineering needs, and help integrate the ones that prove useful
• Develop visibility into which tools and workflows are useful, where they create friction, and how they should evolve
• Create clear guidance and documentation that helps engineers adopt new AI-enabled development practices
• Improve code review workflows by using feedback from real engineering work to identify and address recurring friction
• Influence the foundations of an autonomous software factory, including systems that help agents complete bounded engineering tasks safely and effectively
• Work directly with engineers and internal users to understand their needs, validate solutions, and make thoughtful product tradeoffs
• Help the team move ambiguous, dependency-heavy projects from early exploration through reliable delivery
• Raise the quality bar through thoughtful code reviews, technical design discussions, testing, observability, and attention to reliability and security
• Partner closely with engineers and cross-functional stakeholders while contributing as a dependable, collaborative teammate