GPU Infrastructure Engineer

Rune connects stranded renewable energy, particularly from solar and wind power plants, to high-performance computing through modular micro data centers. The GPU Infrastructure Engineer will own hands-on, customer-facing technical support for GPU clusters, including diagnosing networking, node provisioning, GPU health, driver and firmware compatibility, and production incidents.

Responsibilities:

• Serve as the primary technical responder for customer-reported networking, GPU-interconnect, and node issues on live clusters
• Diagnose multi-node NCCL and RDMA performance regressions, including issues sourced to GPUDirect RDMA contention, and separate confirmed findings from working hypotheses when talking to customers
• Configure and validate RoCEv2 congestion control (ECN/DCQCN, PFC, DCBX) and reconcile host-side NIC settings against the switch fabric's actual behavior
• Diagnose GPU hardware faults using DCGM telemetry and Xid error codes, distinguishing genuine hardware failures (RMA) from driver, firmware, or configuration issues
• Own driver and firmware version compatibility across the fleet: GPU driver, NIC firmware (OFED/MOFED), kernel, and validate compatibility before and after updates
• Provision, re-provision, and decommission GPU nodes: power-cycling, secure wipe, re-imaging, BIOS/NUMA/PCIe topology validation, and acceptance testing before handoff to a customer
• Triage edge-network and jumpbox access issues affecting customer provisioning workflows (e.g., automated blocking of legitimate high-concurrency SSH/Ansible traffic)
• Write clear incident updates and post-resolution summaries for customer engineering teams, and feed recurring issues back into provisioning defaults and acceptance-test procedures