Applied AI Evaluation Engineer

BMO's Applied AI team is dedicated to establishing high-performing AI systems within the bank. The Applied AI Evaluation Engineer will design and optimize evaluation frameworks and pipelines to ensure the quality and performance of AI models, collaborating with various teams to integrate evaluation standards into scalable solutions.

Responsibilities:

• Develop, implement, and maintain evaluation pipelines and harnesses for AI models, ensuring reliability, reproducibility, and scalability for banking applications
• Collaborate with the Product Owner to translate evaluation standards and business requirements into technical solutions
• Prototype and validate evaluation methods using real banking workflows, integrating feedback into model training and deployment
• Integrate evaluation metrics and signals into research, training, and production systems to drive measurable improvements in model performance and customer outcomes
• Partner with engineering, research, and product teams to shape model interaction paradigms and deployment strategies
• Build reusable tools and systems that enable contributions from across BMO and raise the quality bar for AI solutions
• Ensure all evaluation frameworks adhere to regulatory, compliance, and Responsible AI requirements
• Document processes, share knowledge, and contribute to a culture of continuous learning and responsible innovation