Lead the AI evaluation discipline within the engineering organization
Design and implement evaluation methodologies for LLM and AI systems
Design statistical evaluation methods for AI model performance
Collaborate on data pipelines, feature engineering, and context design
Contribute to LLMOps / MLOps practices (CI/CD, monitoring, lifecycle management)
Build and deploy LLM, RAG, and agent-based solutions
Integrate AI systems into existing platforms and data ecosystems
Lead technical discussions and mentor team members

5+ years in AI Engineering, Machine Learning, or Data Science
Strong experience with Python and modern AI frameworks
Hands-on experience with LLMs, RAG, or agent-based systems
Understanding of cloud platforms (Azure, AWS, or GCP)
Experience building production-grade systems
Strong problem-solving and communication skills.
Experience with AI evaluation frameworks or benchmarking approaches (Nice to Have)
Familiarity with observability tools (e.g. Langfuse, tracing, monitoring) (Nice to Have)
Experience with MLflow or experiment tracking tools (Nice to Have)
Knowledge of vector databases, semantic search, or knowledge graphs (Nice to Have)
Experience with Azure OpenAI or similar platforms (Nice to Have)
Background in statistics or experimentation design (Nice to Have)
Experience mentoring or leading engineers (Nice to Have).

AI Engineer

Key skills