ML Infrastructure Engineer, Training

Dyna Robotics is a cutting-edge company focused on AI-driven robotics, founded by experienced entrepreneurs and backed by significant investment. They are seeking a ML Training Infrastructure Engineer to architect and build systems that optimize their multi-cloud GPU fleet for training purposes, ensuring efficient utilization and performance for researchers.

Responsibilities:

• Scale Distributed Training: Architect and own the infrastructure for large-scale GPU clusters. You’ll implement sharding, activation checkpointing, and memory optimization (ZeRO, FSDP) to enable the training of massive multimodal models
• Optimize Researcher Ergonomics: Build a research codebase and job scheduling system (Kubernetes/SLURM) that prioritizes fast iteration, automated retries, and seamless failure recovery
• High-Performance Data Handling: Design high-throughput pipelines to ingest and transform terabytes of multimodal robot data (video, proprioception, 3D signals), ensuring dataloaders never starve the GPUs
• Production Inference: Build low-latency inference pipelines for real-time robot control. You’ll apply quantization, distillation, and model compilation (TensorRT, Triton) to move models from the lab to the physical world
• Deep Systems Profiling: Dive into the weeds of GPU utilization, I/O bottlenecks, and memory fragmentation to squeeze every bit of performance out of our expanding compute fleet