Senior AI Engineer - Model Selection and Orchestration

Zip is a global Buy Now, Pay Later company that provides simpler and fairer payment options to millions of customers. The Senior AI Engineer will build production AI capabilities, including decision and orchestration layers that select models, tools, retrieval systems, or deterministic workflows. The role also involves evaluating model performance, integrating APIs and tools safely, and improving reliable AI platform capabilities across customer-facing and internal experiences.

Responsibilities:

• Design and implement the decision layer that determines how an agent should respond to a task, including when to use deterministic logic, retrieval, smaller models, reasoning models, tools, or human escalation
• Develop intent classification and clarification capabilities that distinguish information requests from actions, identify ambiguous requests, evaluate confidence, and determine when additional information or escalation is required
• Benchmark rules, retrieval approaches, classifiers, smaller models, and reasoning models against representative tasks. Use measurable outcomes such as task success, quality, latency, safety, and total inference cost to recommend model-selection decisions
• Implement and improve model adapters, routing logic, structured outputs, timeouts, retry strategies, fallback behavior, and observable failure handling. Build orchestration that is testable and avoids unnecessary dependency on any single model provider
• Connect agents to approved APIs and tools with appropriate authentication, authorization, policy checks, and customer confirmation. Design for duplicate requests, idempotency, recovery, and other failure scenarios when agents perform state-changing actions
• Partner with Security and Risk to implement approved input and output protections, sensitive-data handling, prompt-injection defenses, and controlled access. Maintain clear separation between model judgment and the permissions required to execute actions
• Develop reusable orchestration patterns, model adapters, routing components, and tooling that make it easier for domain engineering teams and agent builders to create reliable AI-powered experiences
• Partner with evaluation engineering to test changes before release, monitor system behavior, investigate failures, and continuously improve model selection and orchestration using production evidence
• Use AI-assisted development tools as a core part of your engineering workflow to accelerate development, troubleshoot problems, improve code quality, and write and maintain tests. Apply sound engineering judgment when evaluating AI-generated solutions for correctness, security, performance, and maintainability

Requirements:

• Minimum of 10 years of related experience with a Bachelor's degree; or 6 years and a Master's degree; or a PhD with 3 years experience; or equivalent experience
• 10+ years of professional experience building, testing, deploying, and operating production software, including hands-on delivery of AI or ML-enabled applications beyond prototypes
• Strong experience with Python or TypeScript and modern software engineering practices, including API design, automated testing, distributed services, and debugging across application, model, and tool boundaries
• Practical experience building applications using LLM tool calling, structured outputs, retrieval-augmented generation, and agent or workflow orchestration
• Ability to evaluate classification quality, uncertainty, fallback strategies, and different model or system approaches using measurable outcomes such as task success, latency, reliability, and total inference cost
• Experience with cloud deployment, automated testing, CI/CD, observability, identity and access controls, secrets management, retries, idempotency, and failure recovery in production systems
• Strong judgment about when a problem should be solved with deterministic software, retrieval, a model, or a combination of approaches, as well as when an agent should ask for clarification or hand work to a person
• Ability to independently deliver complex technical work within shared architecture standards while collaborating effectively with Principal and Staff Engineers, Product, domain engineering teams, Security, and Risk
• You communicate tradeoffs clearly, contribute thoughtful code reviews, and help strengthen engineering practices across the team
• Hands-on experience using AI-assisted development tools as part of your day-to-day engineering workflow to develop, debug, test, evaluate, and improve software while applying sound engineering judgment to AI-generated solutions
• Experience with Google Cloud, Google ADK, Model Garden, or equivalent managed AI platforms; semantic routing; open-weight models; Model Context Protocol (MCP); reusable agent-development frameworks; Azure integrations; or payments, fintech, or other environments where correctness, auditability, and controlled access are essential