Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
Close the production feedback loop by mining failure patterns from real traffic and proposing fixes
Harden methodology: judge stability and non-determinism handling
Translate numbers into decisions in weekly syncs with clear trade-offs

Python and SQL — you can build an analysis end-to-end
Solid foundation in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is
Analytical mindset — you start from the business question, not from the tool
3+ years in analyst / data scientist roles, at least one in a product context
Experience in quality analytics for ML systems is a plus
Hands-on experience evaluating LLM applications is a plus
Experience building LLM agents is a plus

Make a genuine impact on the product
Join our upward trajectory, and grow with us
Provide resources and opportunities for continuous personal and professional development
Enjoy flexibility of traveling and working remotely or in a hybrid model across Europe
Become a stock options holder through Stock Options Program
Receive unwavering support and care from Finom
Participate in exclusive Work & Swim Program
Modern, friendly, and eco-conscious corporate culture
Equal Opportunity employer valuing diversity

Product Data Scientist – AI Evaluation, Quality

Key skills