Remote | Machine Learning Research Engineer — $55–$85/hour

24-MAG is offering a specialized full-time consulting opportunity for machine learning engineers and research practitioners. The role focuses on developing advanced evaluation benchmarks for AI systems, requiring hands-on experience with ML models, task design, experimentation, and collaboration with researchers.

Responsibilities:

• Turn practical ML research ideas into well-defined, multi-step evaluation tasks
• Develop assignments involving model training, experimental modifications, and performance analysis
• Define clear technical requirements, expected outputs, and success criteria
• Ensure tasks assess genuine implementation and experimental reasoning rather than superficial library usage
• Implement reference solutions using Python, scripts, and notebook environments
• Configure and run model-training experiments from setup through final evaluation
• Modify model components, training procedures, reward functions, or experimental parameters
• Validate code, dependencies, datasets, intermediate outputs, and final results
• Document complete workflows so experiments can be reproduced independently
• Review how frontier AI models approach complex machine learning tasks
• Assess implementation quality, experimental methodology, and technical conclusions
• Identify coding errors, unsupported assumptions, weak experimental controls, and misleading interpretations
• Determine whether reported improvements are supported by the observed results
• Explain clearly where and why a model-generated solution fails
• Develop selected tasks involving reinforcement learning fundamentals
• Evaluate reward-function changes, policy-training behaviour, and experimental outcomes
• Assess whether proposed modifications produce the intended training effect
• Identify instability, unintended incentives, or incorrect interpretations of RL results
• Work closely with researchers, task authors, and fellow machine learning specialists
• Compare evaluation decisions to maintain consistent and rigorous benchmark standards
• Refine task instructions, reference solutions, and grading criteria based on testing outcomes
• Share recurring model failure patterns and opportunities for stronger benchmark coverage

Requirements:

• At least 1 year of experience in machine learning research, research engineering, or a comparable technical role
• Hands-on experience training and evaluating ML models through complete experimental workflows
• Strong understanding of experiment setup, execution, analysis, and reproducibility
• Familiarity with large language model capabilities, limitations, and evaluation techniques
• Working proficiency in Python and Git
• Comfort using both scripting and notebook-based environments
• Strong technical writing, analytical reasoning, and attention to detail
• Ability to work independently through ambiguous, open-ended research problems
• Reliable availability for approximately 35 hours per week
• A master's degree or PhD in machine learning, computer science, artificial intelligence, engineering, mathematics, or another relevant STEM discipline is highly relevant
• Equivalent practical experience in a research-intensive machine learning role may also be considered
• Academic or professional work involving model training, experimentation, or ML systems may strengthen an application
• Publications, open-source contributions, technical reports, or substantial research projects may also be valuable
• Understanding of reinforcement learning concepts, including reward functions and policy training
• Experience in AI training, model evaluation, or benchmark development
• Background authoring technical tasks, reference solutions, or grading rubrics
• Familiarity with agentic AI systems and multi-step model evaluations
• Experience diagnosing model-training failures or unexpected experimental behaviour
• Knowledge of experimental design, ablation studies, and performance comparison
• Experience reviewing code, notebooks, or research analyses prepared by other practitioners
• Familiarity with reproducible ML environments and collaborative Git workflows