Product Data Scientist – AI Evaluation, Quality

Role Overview

• Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
• Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
• Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts
• Harden methodology: judge stability, non-determinism handling
• Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake

Requirements

• Python and SQL — you can build an analysis end-to-end
• Solid foundation in statistics — sampling, hypothesis testing, variance
• Analytical mindset — you start from the business question, not from the tool
• 3+ years in analyst/data scientist roles, at least one in a product context

Tech Stack

• Python
• SQL

Benefits

• Make a genuine impact on the product
• Join our upward trajectory and grow with us
• Work in the EU with flexible traveling and working remotely or in a hybrid model across Europe
• Become a stock options holder through our Stock Options Program
• Receive unwavering support and care reflecting our commitment to your well-being and success
• Work & Swim program with a month in a corporate apartment in Cyprus
• Equal Opportunity Statement embracing diversity