Zello is a voice-first communication platform that enhances collaboration and productivity for desk-less workers. The Applied AI Engineer will be responsible for building AI agents from prototype to production, monitoring their quality, and driving continuous improvements.
Responsibilities:
- Shipped at least 3 production-grade AI agents within your first 90 days that internal teams actively use (Slack-integrated agents, workflow automations, data-driven assistants)
- Built evaluation harnesses for deployed agents with automated quality scoring and regression detection
- Integrated AI tools with Zello's existing systems (Slack, Jira, HubSpot, Snowflake) via APIs, with proper logging and monitoring in place
- Established reusable code patterns and component libraries that make future agent development faster
- Taken ownership of deployed agent operations: monitoring performance, overseeing human reinforcement workflows, triaging failures, and driving measurable improvement in agent quality over time
- Independently scoped and shipped AI tools for new use cases, whether identified by stakeholders or discovered on your own
- Build AI agents and automations end-to-end: from scoping the use case through deployment and ongoing maintenance
- Write production Python code that integrates LLM APIs (prompt construction, response handling, context management, tool use) into real workflows
- Connect AI tools with Zello's systems (Slack, Jira, HubSpot, Snowflake) through APIs, handling authentication, rate limits, error cases, and logging
- Monitor deployed agents in production: track quality metrics, triage failures, and ship improvements based on real usage data
- Manage human reinforcement operations: review agent outputs, maintain feedback loops, and tune agent behavior based on reinforcement signals
- Build and maintain evaluation harnesses that catch regressions and measure agent quality programmatically
- Create reusable components, patterns, and documentation that raise the bar for future development on the team
- Communicate clearly with technical and non-technical stakeholders about what you've built, what's working, and where things need attention
Requirements:
- 2-5 years of professional experience in software engineering, AI engineering, or a related technical role
- Written production Python code and can point to real things built with it: tools, integrations, automations, shipped products
- Understanding of LLM APIs at a practical level, including prompt construction, context management, and tool use
- Ability to decompose messy problems into clean components with well-defined interfaces
- Experience integrating systems via APIs, including reading API docs, handling authentication, managing rate limits, and dealing with edge cases
- Quality instinct, with a focus on monitoring and testing to ensure systems are working post-deployment
- Comfort with operational ownership, including monitoring, reviewing outputs, and maintaining AI systems in production
- Ability to quickly pick up new frameworks, APIs, and domains
- Clean and documented code that others can read and understand