Sr. AI QA Engineer / Lead

Looking for a AI Quality Engineering Lead with expertise in LLMs (OpenAI, Azure OpenAI), Agentic AI, RAG pipelines, Python automation, and AI validation frameworks, responsible for driving AI quality assurance, workflow optimization, intelligent test automation, runtime reliability, and scalable enterprise AI governance across healthcare-focused AI solutions.

Roles and Responsibilities: -

Quality Engineering Architecture & Workflow Optimization Lead architecture and technical implementation of AI Quality Engineering solutions supporting AI-powered applications, LLM-enabled workflows, intelligent automation solutions, agentic systems, and enterprise AI platforms.

Design and implement scalable AI validation frameworks, AI-assisted testing approaches, runtime quality controls, reusable testing accelerators, and workflow optimization capabilities supporting enterprise AI delivery initiatives.

Support modernization of traditional Quality Engineering practices through intelligent automation, workflow orchestration, and scalable quality engineering patterns.

Develop automated validation approaches, anomaly detection processes, testing pipelines, and quality metrics supporting governance-aligned AI deployment practices.

Design and optimize human-in-the-loop validation workflows, operational review processes, and AI quality assurance controls supporting reliable and scalable AI-enabled systems.

Partner with engineering and business teams to identify operational bottlenecks, workflow optimization opportunities, automation use cases, and scalable quality engineering improvements.

Experience: -

8+ Years

Location: -

Eden Prairie, MN(Hybrid)

Duration: -

6 Months Contract with possible extension

Educational Qualifications: -

Engineering Degree BE/ME/BTech/MTech/BSc/MSc.

Technical certification in multiple technologies is desirable.

Mandatory skills

AI Validation, Runtime Assurance & Automation Support AI validation activities, including prompt testing, workflow testing, regression testing, runtime quality assurance, and production reliability support. Partner with AI Engineering, AIOps, LLMOps, Security, Governance, Clinical, and Data teams to support scalable AI Quality Engineering and workflow automation processes across enterprise AI initiatives.

Design and support runtime quality practices, including telemetry alignment, monitoring coordination, validation processes, and runtime reliability improvement efforts.

Drive adoption of AI-assisted testing approaches, intelligent automation, reusable testing accelerators, and orchestration-aware testing practices.

Support observability and runtime visibility initiatives improving reliability, traceability, and confidence across AI-enabled systems.

Collaborate with Clinical, Operational, and Engineering stakeholders to support validation of healthcare workflows, operational processes, and AI-enabled business solutions.

Technical Enablement, Delivery & Operational Support delivery coordination activities across AI Quality Engineering and workflow optimization initiatives, including implementation planning, issue tracking, operational support, and release coordination activities.

Partner with stakeholders to evaluate implementation readiness, workflow dependencies, operational risks, automation opportunities, and quality considerations for AI initiatives. Support tooling evaluations, automation frameworks, orchestration tooling, and modernization initiatives supporting AI Quality Engineering maturity.

Help establish reusable workflow automation patterns, scalable testing assets, and engineering enablement practices across delivery teams. Support adoption of modern AI Quality Engineering and workflow optimization practices across engineering and business organizations.

Leadership, Collaboration & Continuous Improvement Lead and mentor engineers, analysts, contractors, and delivery teams while fostering a collaborative, continuously learning, and engineering-focused culture.

Communicate implementation risks, workflow optimization opportunities, technical trade-offs, and operational recommendations to technical and business stakeholders. Promote engineering discipline, continuous improvement, responsible AI adoption, and operational accountability across AI Quality Engineering initiatives.

Skills

Quality LLM, OpenAI,Azure, Python RAG Pipeline, AgenticAI.