AI Conversation Quality Analyst - English & Mandarin

The UVA VEC is seeking bilingual professionals fluent in English and Mandarin to support the evaluation and improvement of advanced conversational AI systems. In this role, you will review and analyze AI-generated responses to ensure their accuracy and clarity, providing feedback to refine communication across various scenarios.

Responsibilities:

• Evaluate AI-generated responses to determine how effectively they address user questions
• Verify information accuracy using reliable public sources and research tools
• Provide structured feedback highlighting strengths, weaknesses, and factual errors
• Assess response quality including reasoning, clarity, tone, and completeness
• Ensure AI outputs align with conversational standards and system guidelines
• Apply consistent evaluation frameworks using established benchmarks and annotation guidelines