Senior Software Engineer, Machine Learning Infrastructure & Automation

fal builds infrastructure, tools, and model access for generative AI products. The Senior Software Engineer will develop and maintain ML automation, CI/CD infrastructure, and developer tooling to help teams test, validate, benchmark, and deploy generative AI models reliably and efficiently.

Responsibilities:

• Own ML CI/CD infrastructure Design, build, and maintain automated testing, validation, and deployment pipelines for our ML models and inference services
• Accelerate development cycles.Dramatically reduce CI execution times through intelligent parallelization, caching, test selection, and efficient use of compute resources
• Build automated model validation.Develop systems that test model outputs, detect quality regressions, and validate changes across different models, GPU architectures, and configurations
• Automate performance benchmarking. Build continuous performance testing that detects regressions in inference latency, throughput, GPU utilization, and cost
• Build automated pricing and deployment checks. Ensure model pricing, billing configurations, API schemas, and deployments are validated automatically before reaching production
• Extend our agentic engineering workflows. Develop AI-powered automation and agentic coding systems that automatically diagnose CI failures, identify regressions, propose fixes, and streamline engineering workflows
• Improve deployment reliability. Build automated safeguards, deployment verification, rollback mechanisms, and monitoring to ensure new model releases are reliable
• Eliminate engineering toil. Identify repetitive tasks across the ML team and build tools and systems that automate them, allowing engineers to focus on developing new models and improving performance

Requirements:

• 5+ years of software engineering background with proficiency in Python and experience building production infrastructure and developer tooling
• Experience designing and operating CI/CD systems using GitHub Actions or comparable technologies
• Deep understanding of automated testing, build systems, dependency management, caching, and parallel execution
• Experience working with containerized workloads, Docker, and cloud infrastructure
• Ability to design reliable distributed systems and debug complex infrastructure failures
• Strong understanding of observability, including logs, metrics, tracing, and automated alerting
• A passion for developer productivity and a demonstrated ability to eliminate manual processes through automation
• Comfortable working independently, identifying high-impact problems, and building end-to-end solutions
• Experience with ML infrastructure, PyTorch, GPU workloads, or model-serving systems
• Familiarity with NVIDIA GPU architectures and multi-GPU environments
• Experience building automated inference benchmarks or ML quality evaluation frameworks
• Experience with agentic coding tools such as Codex or Claude Code, or building custom AI engineering agents
• Experience optimizing CI/CD pipelines at scale, including distributed test execution and ephemeral compute environments
• Experience developing internal developer platforms or infrastructure-as-code tooling