Define and drive the long-term strategy for search relevance, retrieval evaluation, ranking optimization, and AI system quality.
Lead initiatives focused on improving:
Search relevance and ranking quality.
Semantic retrieval and vector search
Retrieval-augmented generation (RAG)
AI grounding and hallucination mitigation
User discovery and engagement outcomes
Establish scalable evaluation methodologies for search, retrieval, recommendation, and LLM-powered systems.
Guide experimentation and optimization across lexical, semantic, hybrid, and AI-assisted retrieval architectures.
Partner with Product and Engineering leadership to align search and AI investments with customer and business priorities.
Influence technical direction for retrieval systems, evaluation infrastructure, and AI quality frameworks across platforms.
Define and operationalize evaluation frameworks for search and generative AI systems.
Build scalable processes for benchmark creation, annotation quality, evaluation governance, and performance reporting.
Drive rigorous, evidence-based decision-making across AI and search initiatives.
Champion responsible AI practices focused on quality, reliability, trust, and measurable user impact.
Lead, mentor, and grow a high-performing team of Data Scientists and Analysts.

Master’s or PhD in Computer Science, Data Science, Machine Learning, Information Retrieval, Statistics, NLP, or a related quantitative field.
10+ years of experience in Data Science, Machine Learning, Information Retrieval, Search Relevance, Evaluation Systems, or Applied AI.
Significant experience leading and scaling high-performing technical teams in complex, cross-functional organizations.
Deep expertise in:
Search relevance and ranking systems.
Information retrieval and semantic search
Retrieval-augmented generation (RAG)
Evaluation methodologies for IR and generative AI systems
Experimentation frameworks and A/B testing
Strong experience with:
Vector retrieval and hybrid search architectures.
LLM evaluation and AI quality measurement
Embeddings, reranking, and retrieval orchestration
Evaluation datasets, benchmarking, and annotation workflows
Advanced programming skills in Python.
Experience with modern AI/ML frameworks and tooling (e.g., PyTorch, Hugging Face, LangChain, LangGraph, Haystack).
Experience working with large-scale datasets, distributed data/ML platforms, and production AI systems.
Strong understanding of statistical analysis, experimentation design, and evaluation science.
Excellent communication and stakeholder management skills, including experience influencing senior leadership.
Demonstrated ability to balance strategic leadership with pragmatic execution in rapidly evolving AI environments.

Director, Search & AI Evaluation

Key skills