Technical Solution Management Specialist - AI Evaluation, Research, and Technical Architecture Func.

Microsoft is a leading technology company that empowers customers to achieve more through innovative solutions. The Technical Solution Management Specialist will play a key role in establishing the XIO AI Innovation Lab, focusing on AI evaluation, research, and technical architecture to drive AI transformation for customers. This position involves strategic planning, technical roadmapping, and cross-functional program management to ensure effective evaluation of AI opportunities.

Responsibilities:

• Define and communicate a future-state vision for AI evaluation, research quality, and technical architecture across the XIO AI Innovation Lab
• Develop and maintain technical blueprints for research, validation, evaluation, experimentation, and Test Kitchen-ready handoff capabilities
• Define and maintain related processes and playbooks for scaling pre 0-to-1 motions using the lab flow: Signal -> Research -> Validation -> Research Paper -> PoC Candidate -> Test Kitchen
• Influence technical roadmaps for consistent product evaluation frameworks and tooling
• Track best practices, Microsoft product updates, frontier AI research, and emerging AI trends to inform lab strategy and publication direction
• Ensure alignment of evaluation frameworks with business priorities and Responsible AI standards
• Build and scale multi-dimensional evaluation approaches for diverse lab-stage AI workloads, including chatbots, customer self-service agents, predictive models, and generative tools
• Develop automated testing frameworks, including synthetic scenario generation, adversarial stress-testing, and batch evaluation
• Implement lab-appropriate experimentation patterns, including offline evaluation, synthetic scenario testing, feature-flagged trials where appropriate, and monitoring plans for downstream PoC owners
• Partner with engineering, data science, and prototyping teams to integrate evaluation tools into prototype scaffolds, CI/CD pipelines where relevant, and Test Kitchen handoff plans
• Develop reference architectures, data needs assessments, secure data pipeline considerations, hosting-model options, and reusable solution patterns for candidate AI solutions
• Package validated research into PoC candidate artifacts with value proposition, evidence, architecture, evaluation plan, governance considerations, and owner model
• Coordinate across business, technical, and governance teams to ensure successful delivery of evaluation programs
• Help establish and operate the XIO AI Innovation Lab as a lightweight research and validation capability, acting as a consultant and trainer for project teams
• Drive organizational adoption of best practices, shared infrastructure, common metrics, research publication standards, and Test Kitchen handoff discipline
• Support focused leadership alignment, research reviews, Demo Showcase updates, and handoffs to prototyping or product teams without slowing early-stage research autonomy
• Apply UX research principles to understand lived experiences and translate insights into actionable improvements for AI solutions
• Collaborate with domain experts, support engineers, CSU/account managers/CSAMs, and business owners to define success criteria and evaluation rubrics
• Lead direct customer and field-team engagements and collect performance metrics to inform continuous improvement and readiness for PG engagement
• Integrate Responsible AI evaluations, safety, ethics, and compliance checks into all testing and experimentation processes
• Manage gate reviews, documentation, risk findings, and RAI oversight appropriate to lab-stage research and Test Kitchen handoff readiness
• Foster a culture of "fail fast, learn fast" with accountability and continuous improvement

Requirements:

• Bachelor's Degree AND 4+ years experience in technical program management, preferably in AI, machine learning, research, data science, technical architecture, or customer experience domains
• OR equivalent experience
• 2+ years of experience managing cross-functional and/or cross-team projects
• Bachelor's Degree AND 8+ years experience in technical program management, preferably in AI, machine learning, research, data science, technical architecture, or customer experience domains
• OR product development
• 6+ years of experience managing cross-functional and/or cross-team projects
• 1+ year(s) of experience reading and/or writing code (e.g., sample documentation, product demos)
• Proven track record of AI evaluation, testing methodologies, and performance metrics
• Experience with automated testing pipelines, synthetic data generation, and A/B experimentation
• Familiarity with Responsible AI frameworks, governance, and compliance standards
• Demonstrated ability to produce research papers, decision-grade technical artifacts, reference architectures, or reusable solution patterns
• Ability to adapt strategy based on emerging technology trends, frontier AI research, customer signals, and industry research
• Comfort operating in ambiguous pre 0-to-1 environments and moving work from research to prototype-ready handoff