Remote | Machine Learning Systems Evaluation Engineer — Up to $90/hour

24-MAG is offering a specialized remote consulting opportunity for experienced machine learning engineers. The role focuses on evaluating complex machine learning and AI engineering implementations, supporting workflows related to ML system evaluation, and providing structured feedback on MLOps and deployment processes.

Responsibilities:

• Use modern coding agents to complete and evaluate complex machine learning and AI engineering tasks
• Review generated implementations involving model training, inference systems, MLOps workflows, LLM applications, and AI-powered product features
• Assess technical outputs for correctness, quality, maintainability, performance, reliability, and production-readiness
• Apply professional machine learning engineering judgment to realistic technical scenarios
• Evaluate ML system workflows involving model deployment, inference infrastructure, monitoring, testing, and production integration
• Review implementation choices related to scalability, latency, data flow, model serving, reliability, and system maintainability
• Identify bugs, edge cases, performance issues, failure modes, and weak assumptions in ML engineering outputs
• Provide structured feedback on MLOps design, deployment patterns, and production ML system quality
• Compare outputs from multiple coding agents and assess their strengths, weaknesses, accuracy, and practical usefulness
• Identify where generated solutions succeed, where they fail, and where additional ML engineering judgment is required
• Evaluate whether generated machine learning implementations reflect real-world engineering standards
• Document technical review findings clearly for project teams and quality evaluation workflows
• Produce clear, structured evaluations of machine learning engineering tasks and generated outputs
• Explain reasoning around model training, inference systems, deployment infrastructure, LLM applications, performance, and architectural trade-offs
• Support technical assessment workflows by documenting accepted work, improvement areas, and practical engineering conclusions
• Help ensure outputs reflect production-scale machine learning engineering expectations

Requirements:

• 2+ years of professional machine learning engineering experience
• Hands-on experience building production ML systems, model deployment infrastructure, LLM applications, or AI-powered products
• Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or comparable tools
• Ability to evaluate generated machine learning implementations and identify technical trade-offs, bugs, edge cases, and performance issues
• Strong understanding of model training, inference workflows, MLOps, data pipelines, evaluation methods, deployment patterns, and system reliability
• Clear written communication skills and comfort documenting technical reasoning in a remote, project-based environment
• A degree in Computer Science, Machine Learning, Artificial Intelligence, Data Science, Software Engineering, Computer Engineering, Statistics, Mathematics, or a related technical field is helpful
• Equivalent professional experience in machine learning engineering, applied AI, MLOps, LLM applications, or production ML systems is also highly relevant
• Experience deploying ML systems to production is strongly preferred
• Experience with Python, PyTorch, TensorFlow, scikit-learn, Hugging Face, LangChain, LlamaIndex, MLflow, Ray, or comparable ML tools
• Familiarity with model serving, feature pipelines, vector databases, embeddings, retrieval systems, LLM application architecture, or evaluation frameworks
• Experience with cloud platforms, Docker, Kubernetes, CI/CD pipelines, observability tooling, or production deployment workflows
• Background in technical code review, ML architecture review, model performance evaluation, or large-scale AI product engineering
• Strong comfort working in sprint-based project environments with focused technical assessment windows