Applied AI Engineer

Zello is a voice-first communication platform that enhances collaboration and productivity for desk-less workers. The Applied AI Engineer will be responsible for building AI agents from prototype to production, monitoring their quality, and driving continuous improvements.

Responsibilities:

• Shipped at least 3 production-grade AI agents within your first 90 days that internal teams actively use (Slack-integrated agents, workflow automations, data-driven assistants)
• Built evaluation harnesses for deployed agents with automated quality scoring and regression detection
• Integrated AI tools with Zello's existing systems (Slack, Jira, HubSpot, Snowflake) via APIs, with proper logging and monitoring in place
• Established reusable code patterns and component libraries that make future agent development faster
• Taken ownership of deployed agent operations: monitoring performance, overseeing human reinforcement workflows, triaging failures, and driving measurable improvement in agent quality over time
• Independently scoped and shipped AI tools for new use cases, whether identified by stakeholders or discovered on your own
• Build AI agents and automations end-to-end: from scoping the use case through deployment and ongoing maintenance
• Write production Python code that integrates LLM APIs (prompt construction, response handling, context management, tool use) into real workflows
• Connect AI tools with Zello's systems (Slack, Jira, HubSpot, Snowflake) through APIs, handling authentication, rate limits, error cases, and logging
• Monitor deployed agents in production: track quality metrics, triage failures, and ship improvements based on real usage data
• Manage human reinforcement operations: review agent outputs, maintain feedback loops, and tune agent behavior based on reinforcement signals
• Build and maintain evaluation harnesses that catch regressions and measure agent quality programmatically
• Create reusable components, patterns, and documentation that raise the bar for future development on the team
• Communicate clearly with technical and non-technical stakeholders about what you've built, what's working, and where things need attention

Requirements:

• 2-5 years of professional experience in software engineering, AI engineering, or a related technical role
• Written production Python code and can point to real things built with it: tools, integrations, automations, shipped products
• Understanding of LLM APIs at a practical level, including prompt construction, context management, and tool use
• Ability to decompose messy problems into clean components with well-defined interfaces
• Experience integrating systems via APIs, including reading API docs, handling authentication, managing rate limits, and dealing with edge cases
• Quality instinct, with a focus on monitoring and testing to ensure systems are working post-deployment
• Comfort with operational ownership, including monitoring, reviewing outputs, and maintaining AI systems in production
• Ability to quickly pick up new frameworks, APIs, and domains
• Clean and documented code that others can read and understand