Key skills
AICollaboration
About this role
Role Overview
- Evaluate AI outputs related to legal and compliance content
- Perform structured scoring, comparison, classification, and judgment tasks
- Assess accuracy, relevance, completeness, risk, and alignment with legal norms
- Identify hallucinations, ambiguity, unsafe guidance, or misleading outputs
- Apply domain-specific guidelines consistently across tasks
- Validate and refine evaluation rubrics and edge-case handling
- Perform adjudication where raters disagree
- Conduct error analysis and qualitative reviews of model behavior
- Partner with LILT research, product, and customer teams on evaluation design
- Support red-teaming, risk analysis, and model readiness assessments
Requirements
- Legal practitioner, compliance professional, legal analyst, or policy specialist
- Experience interpreting statutes, regulations, contracts, policies, or legal guidance
- Strong attention to detail and comfort working with structured evaluation criteria
- Native or professional fluency in one or more supported languages is required
- English fluency is required for guidelines, feedback, and collaboration.
Benefits
- Contract-based, flexible participation
- Project-based work with clear expectations and timelines
- Opportunities for recurring work based on performance and demand