Product Manager II

Microsoft is a leading technology company committed to empowering individuals and organizations. As a Product Manager II on the Copilot Offline Evaluation Platform team, you will define and deliver capabilities to streamline evaluation workflows while partnering with engineering and data science teams.

Responsibilities:

• Own Product Areas Within the Evaluation Lifecycle
• You will drive strategy, execution, and customer outcomes for one or more pillars of COEP, such as:
• Evalset generation and configuration flows (tenant profiles, grounding data, assertions, ingestion graphs)
• Evaluation environment provisioning and reuse
• Integration with EGP, EMP, and C‑Store orchestrators
• Data/DP Pipeline integration for scenario‑specific query pattern generation
• Agentic UX and natural‑language-driven evaluation tooling enabling end-to-end flow without manual engineering intervention
• Deeply Understand COEP Customer Personas
• You Will Advocate For Users Across The Platform
• Scenario Owners, who define evaluation goals, author prompts/assertions, and drive evaluation demand
• Feature Developers, who execute eval jobs and review metrics without needing deep infra knowledge
• Quality teams and connector owners (e.g., GCO) who depend on precision, recipes, and repeatability for high-quality evaluations
• Drive Clarity & Execution Across Engineering and Partner Teams
• Break down ambiguous, cross-system work into crisp requirements, aligned around customer workflows
• Partner with EGP, EMP, DP Pipeline, and C‑Store engineering teams to ship high‑quality platform capabilities
• Contribute to shared COEP reviews, PM team syncs, and cross-org syncs
• Produce specs, requirements, UX flows, and success metrics for your feature areas
• Deliver Platform Reliability, Quality, and Usability
• Ensure features support determinism, reproducibility, and tenant coherence
• Define MVP criteria, success KPIs, backward compatibility expectations, and instrumentation needs
• Drive E2E dogfooding sessions, bug bashes, and quality validation efforts for major milestones (e.g., COEP Agent MVP)
• Champion Platform Simplification and Scale
• Identify opportunities to reduce friction, automate manual steps, and unify evaluation workflows
• Simplify onboarding and enable self-serve evaluation flows for hundreds of Copilot product teams
• Create documentation, recipes, templates, and customer-ready guidance to accelerate adoption

Requirements:

• Bachelor's Degree AND 3+ years experience in product/service/program management or software development + OR equivalent experience
• Bachelor's Degree AND 8+ years experience in product/service/program management or software development + OR equivalent experience
• 2+ years experience taking a product, feature, or experience to market (e.g., design, addressing product market fit, and launch, internal tool/framework)
• 4+ years experience improving product metrics for a product, feature, or experience in a market (e.g., growing customer base, expanding customer usage, avoiding customer churn)
• 4+ years experience disrupting a market for a product, feature, or experience (e.g., competitive disruption, taking the place of an established competing product)
• Experience with ML/AI systems, evaluation tools, or developer platforms Familiarity with Copilot products and workflows
• Experience building natural‑language or agentic workflows
• Solid systems thinking, execution skills, and technical aptitude