Research Scientist - Pre-training

Pluralis Research develops decentralized training and serving of large AI models across consumer-grade devices. The Research Scientist will build and optimize distributed pretraining systems for frontier-scale models, including communication efficiency, fault tolerance, and run monitoring across heterogeneous hardware and networks.

Responsibilities:

• Distributed pretraining: Implement and optimize model-parallel training. Data, pipeline, and tensor parallelism for large models on heterogeneous GPUs under low-bandwidth, high-latency links
• Performance optimization: Implement techniques that reduce communication overhead while maintaining model convergence in challenging network environments
• Elasticity and fault tolerance: Make runs survive node churn. Robust checkpointing, state synchronization, and recovery as participants join and leave
• Run instrumentation: Build the monitoring that shows throughput, bottlenecks, and model quality across hundreds of devices