Lead AI Engineer, Data Solutions

Role Overview

• Build the Agent Flywheel
• Design feedback loops that enable agents and ML systems to improve from real-world outcomes
• Track outcomes (engagement, conversion, quality) and evaluate agent performance
• Build pipelines that collect and structure agent traces into training and evaluation datasets
• Drive continuous improvement via prompting, policies, model selection, and fine-tuning
• Develop ML & Agent Systems
• Build and deploy ML models (classification, ranking, forecasting, recommendation)
• Design AI agents that combine LLM reasoning, tool usage, and ML decisioning
• Implement reusable patterns for multi-step reasoning, tool orchestration, and structured outputs
• Integrate models and agents into business-critical workflows
• Own Data & Model Pipelines
• Design and build scalable data pipelines (batch and near real-time) for training, evaluation, and inference
• Transform raw interaction data into features, labels, and evaluation datasets
• Enable continuous retraining and evaluation through tightly coupled data + model pipelines
• Ensure data quality, consistency, and reliability
• Evaluation & Experimentation
• Build offline and online evaluation frameworks
• Develop evaluation datasets, golden traces, and regression-style test sets
• Run A/B experiments and track key metrics (quality, revenue impact, latency, etc.)
• Use production signals to drive continuous optimization
• Systems & API Development
• Build scalable Python services and APIs powering agent workflows
• Collaborate with platform teams while owning application-level systems
• Ensure reliability, observability, and performance

Requirements

• 6+ years in AI/ML engineering or applied data science
• Strong Python experience in production systems
• Proven experience building and deploying ML models
• Experience building data pipelines (ETL/ELT, batch or streaming)
• Experience with APIs and backend systems
• Experience with LLM-powered systems (prompting, orchestration, evaluation)
• Familiarity with agent workflows and tool usage
• Experience with evaluation loops, agent traces, or iterative improvement systems preferred
• Experience building data pipelines supporting ML systems
• Familiarity with tools like Spark, Airflow/Dagster, Snowflake/BigQuery
• Understanding of data quality, lineage, and reproducibility
• Strong understanding of supervised learning and evaluation methods
• Experience with A/B testing and experimentation
• Ability to design systems combining ML, LLMs, and business logic.

Tech Stack

• Airflow
• BigQuery
• ETL
• Python
• Spark

Benefits

• time off programs
• medical
• dental
• vision
• mental health support
• paid parental leave
• life and disability insurance
• 401(k)
• employee stock purchasing program