Software Engineer – LLM Code Evaluator | Remote

Crossing Hurdles is seeking a Software Engineer to work on evaluating model generated code trajectories. The role involves reviewing and annotating code solutions, documenting debugging logic, and ensuring technical accuracy in annotations.

Responsibilities:

• Review model generated code trajectories on real world software engineering tasks
• Reconstruct and annotate chain of thought reasoning behind each solution step
• Document debugging logic, decision points, and problem solving strategies
• Use full task context including codebases, issues, and test cases to infer reasoning
• Ensure annotations reflect realistic developer workflows and technical accuracy

Requirements:

• Strong professional software engineering experience
• Bachelor's degree in Software Engineering, Computer Science, or a related field
• Strong proficiency in Python, JavaScript, TypeScript, or similar languages
• Familiarity with Git based workflows including pull requests and issue tracking
• Ability to clearly articulate technical reasoning in structured written form