Perform quality audits on annotated datasets to ensure that they meet established guidelines and quality benchmarks.
Leverage statistical based quality metrics such as F1 score and inter-annotator agreement to evaluate data quality.
Analyse annotation errors, trends, project processes, and project documentation to identify and understand the root cause of errors and propose remediation strategies.
Resolve and analyse edge-case annotations to ensure quality and identify areas for improvement.
Become proficient in using annotation and quality control tools to perform reviews and track quality metrics.
Become an expert in the project specific guidelines and provide feedback for potential clarifications or improvements.
Identify opportunities to use automation to help enhance analytics, provide deeper insights, and improve efficiency.
Develop and maintain up-to-date documentation on quality standards, annotation guidelines, and quality control procedures.
Provide regular feedback that identifies areas for improvement across the annotation pipeline.
Work closely with key project stakeholders and clients to understand project requirements and improve annotation pipelines.
Assist with training annotators, providing guidance, feedback, and support to ensure data quality.
Provide regular updates that highlight data quality metrics, key findings, and actionable insights for continuous process improvements.

Bachelor’s degree in a technical field (e.g. Computer Science, Data Science) or equivalent practical experience.
1+ years of experience as a data analyst with exposure to data quality and/or data annotation
ideally within an AI/ML context.
Familiarity with the basic concepts of AI/ML pipelines and data.
Strong analytical and problem-solving skills with an exceptional eye for detail.
Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams.
Ability to work independently and manage time effectively to meet deadlines.
A strong problem-solver who thinks critically and drives innovation and continuous optimisation.
A quick learner with the ability to work independently in a fast-paced environment.
A strong focus on detail, balanced against strategic priorities.
A positive can-do attitude and the ability to easily adapt to new environments.
Not afraid to speak up.

AI Data Quality Analyst

Key skills