Developing and implementing quality assurance protocols and evaluation frameworks for AI outputs, establishing standards for accuracy, completeness, and alignment with attorney expectations
Translating transactional practice requirements into structured knowledge repositories, prompt templates, and context specifications that optimize AI system inputs for high-quality outputs
Analyzing AI output quality across workflows by identifying patterns, gaps, and opportunities to improve context, inputs, and evaluation criteria
Monitoring and reporting on quality metrics, accuracy measures, and evaluation results for AI-powered workflows, using data-driven insights to guide knowledge asset refinement and context optimization
Conducting research to identify best practices for knowledge curation, context management, and AI evaluation methodologies, while staying current with advances in retrieval-augmented generation and legal AI applications
Protecting and maintaining any highly sensitive, confidential, privileged, financial, and/or proprietary information that Latham & Watkins retains
Requirements
Possess strong expertise in knowledge engineering and AI systems, with the ability to curate and structure information to optimize AI inputs and outputs for accuracy, consistency, and alignment with legal practice standards
Demonstrate a solid understanding of transactional legal practice, including mergers and acquisitions (M&A), finance, and capital markets workflows, with the ability to translate practice expertise into structured knowledge assets and evaluation criteria
Exhibit strong knowledge curation capabilities, including developing reference libraries, context repositories, evaluation datasets, and quality assurance frameworks for AI-powered transactional tools
A bachelor’s degree or equivalent, preferably in law, computer science, information systems, or a related field
A minimum of three (3) years of experience in legal technology, knowledge management, or legal practice, with demonstrated experience in knowledge curation, artificial intelligence (AI) evaluation, or quality assurance
Benefits
Healthcare, life and disability insurance
A generous 401k plan
At least 11 paid holidays per year, and a PTO program that accrues 23 days during the first year of employment and grows with tenure
Well-being programs (e.g. mental health services, mindfulness and resiliency, medical resources, well-being events, and more)
Professional development programs
Employee discounts
Affinity groups, networks, and coalitions for lawyers and staff