Finance Domain AI Evaluator, AI Quality Analyst

Role Overview

• Apply finance expertise to assess the performance of advanced Large Language Models (LLMs) by creating realistic finance-related scenarios.
• Develop expert reference answers and evaluate AI-generated responses against defined quality standards.
• Create realistic, finance-focused prompts and scenarios that reflect real-world professional challenges.
• Develop tasks across multiple finance-related areas such as Financial Analysis, Corporate Finance, Accounting, Investment Analysis, Equity Research, Banking, Risk Management, and Valuation.
• Ensure tasks accurately represent professional finance workflows and decision-making processes.
• Develop high-quality reference answers that demonstrate finance expertise and provide clear reasoning, calculations, assumptions, and conclusions.
• Review and evaluate AI-generated responses from LLMs using structured evaluation criteria.
• Analyze model outputs for accuracy, quality, relevance, and compliance with professional finance standards.
• Provide detailed feedback to support AI model evaluation, benchmarking, and continuous improvement efforts.

Requirements

• Bachelor's degree or higher in Finance, Accounting, Economics, Business Administration, Banking, Investment Management, or a related field.
• Professional experience in finance, accounting, investment banking, corporate finance, equity research, auditing, or related areas.
• Strong understanding of financial concepts, terminology, valuation methodologies, and analytical frameworks.
• Ability to analyze and evaluate complex financial information.
• Strong attention to detail and ability to maintain consistency in quality assessments.
• Excellent written and verbal English communication skills.
• Ability to follow structured guidelines, evaluation frameworks, and quality standards.
• Comfortable working independently in a task-based environment.
• Availability to commit to a full-time schedule of 40 hours per week.
• Experience in Investment Banking, Equity Research, Financial Modeling, Valuation, Corporate Finance, M&A, or Capital Markets (preferred).
• CFA Charterholder or CFA Candidate actively pursuing the CFA designation (preferred).
• Experience in financial modeling, valuation, investment research, auditing, risk analysis, or corporate finance (preferred).
• Familiarity with Generative AI, Large Language Models (LLMs), or AI-powered tools (preferred).
• Previous experience in quality assurance, content evaluation, annotation, auditing, or benchmarking projects (preferred).
• Experience developing assessment frameworks, scoring rubrics, or evaluation criteria (preferred).
• Professional certifications such as CFA, ACCA, CPA, CMA, FRM, CFP, or CA (preferred).