Senior Software Engineer (LLM Evaluation) - Remote

Keystone Recruitment is seeking a Senior Software Engineer to support AI research initiatives focused on large language model training and evaluation. The role involves building high-quality datasets, evaluating model outputs, and designing automated verification frameworks while collaborating with research and engineering teams.

Responsibilities:

• Curate production-grade code examples and build precise reference implementations in:
• Evaluate and refine AI-generated code for efficiency, scalability, correctness, and reliability
• Build automated agents to verify code quality and detect error patterns
• Assess model capabilities across the software engineering lifecycle (architecture design, API design, implementation, launch, monitoring, maintenance)
• Design verification systems that automatically validate software engineering tasks
• Collaborate cross-functionally to benchmark AI coding systems against real-world standards

Requirements:

• 5+ years of professional software engineering experience
• 2+ years of continuous full-time experience at a top-tier product-focused technology company
• Strong expertise in building scalable, production-grade full-stack applications
• Deep understanding of system design, architecture, debugging, and code review best practices
• Experience deploying modern applications using contemporary languages and frameworks
• Excellent written and verbal communication skills, particularly in structured technical evaluations