Review annotated data to ensure it meets Spotify’s quality standards and policies, prioritizing work based on business needs
Deliver high-quality, timely results for Product and Engineering teams using established QA frameworks and metrics such as agreement rates and consensus
Handle complex edge cases, helping define ground truth and reduce ambiguity across datasets
Identify patterns, insights, and areas for improvement, and communicate findings clearly to both technical and non-technical partners
Contribute to feedback loops between annotation teams, R&D collaborators, and content policy experts to improve workflows and outputs
Help develop and refine annotation guidelines, supporting annotator training and continuous improvement
Collaborate closely with teammates across multiple projects and domains

experience working with annotation, data quality, or QA processes in ML/AI environments
familiar with LLM or AI-driven annotation workflows and human-in-the-loop systems
comfortable reviewing large-scale datasets across different modalities such as text, audio, images, or video
care about quality and consistency, with a structured approach to evaluating data
communicate clearly and can explain complex ideas in a simple, accessible way
collaborate well with cross-functional partners in fast-moving environments
solid understanding of the machine learning lifecycle, from data collection to deployment
comfortable working with emerging AI tools and agent workflows
curiosity and interest in music, podcasts, or audiobooks
familiarity with SQL or music metadata standards (nice to have)

Annotation QA Analyst – Content Platform

Key skills