Customer Service & Support AI Rater, Evaluator

Role Overview

• Evaluate AI outputs related to customer service and support interactions
• Perform structured scoring, comparison, classification, and judgment tasks
• Assess accuracy, clarity, tone, helpfulness, and alignment with support best practices
• Identify hallucinations, misleading responses, policy violations, or unsafe guidance
• Apply domain-specific customer support guidelines consistently across tasks
• Validate and refine evaluation rubrics and edge-case handling
• Perform adjudication where raters disagree
• Conduct error analysis and qualitative reviews of model behavior
• Partner with LILT research, product, and customer teams on evaluation design
• Support red-teaming, policy alignment, and model readiness assessments

Requirements

• Customer support professionals, service operations specialists, or CX practitioners
• Experience handling customer inquiries, support workflows, or service escalation
• Strong attention to detail and comfort working with structured evaluation criteria
• Native or professional fluency in one or more supported languages is required
• English fluency is required for guidelines, feedback, and collaboration

Benefits

• Contract-based, flexible participation
• Project-based work with clear expectations and timelines
• Opportunities for recurring work based on performance and demand