Sr. Product Manager, AI

Flodesk is building an AI system to help small business owners create and manage complete marketing campaigns. The Senior Product Manager, AI will shape member-facing AI behavior, own prompts and evaluation practices, and partner with engineering and cross-functional teams to improve the AI experience.

Responsibilities:

• Manage member-facing AI behavior across prompt-to-create, agentic editing and future jobs such as segmentation, scheduling, analytics and recommendations
• Create, test, version and document prompts and agent instructions, with a clear record of what changed, why it changed and how behavior improved
• Define what context, member data, tools and product state the AI needs to do a job well, then translate those needs into clear requirements and acceptance criteria
• Own Flodesk's AI evaluation practice, including representative datasets, behavioral scenarios, scoring rubrics, regression coverage and human review
• Run the recurring evaluation and quality-monitoring process independently using available tools. Set baselines and release gates for prompt, model and agent changes
• Review traces and complete AI interactions to diagnose whether a failure most likely comes from the prompt, context, model, tool, product structure or implementation. Fix what you own and turn the rest into clear, actionable changes for engineering
• Turn member research, support feedback, analytics, production failures and domain expertise into priorities, experiments and durable regression cases
• Own the day-to-day product direction and learning agenda for Flodesk's AI experience. Partner with product, design, marketing, copy and engineering to encode a coherent Flodesk point of view into the system

Requirements:

• You have shaped or meaningfully improved real LLM or agentic products, with evidence that your work improved user outcomes or product quality
• You have practical knowledge of prompts, context, tools, agent behavior and evaluation, and understand how those parts interact in a production experience
• You are hands-on with prompt iteration, evaluation tools and trace review, and are comfortable working with JSON, schemas and structured outputs
• You have strong product judgment and can turn ambiguous member needs into clear behavior, quality definitions and testable hypotheses
• You bring experimental rigor. You can design representative test sets, build useful failure taxonomies and separate signal from anecdote
• You think in systems and reusable patterns. You can make today's email experience better without hard-coding assumptions that make tomorrow's surfaces harder to build
• You communicate clearly across technical and non-technical disciplines and can make sound tradeoffs involving member value, quality, latency, reliability and cost
• Owned AI product behavior, conversation design, computational linguistics or prompt systems
• Built content-generation, creative or marketing tools
• Used Langfuse, Braintrust or comparable evaluation and observability platforms
• Worked on products for small businesses, creators or marketers