xAI is on a mission to create AI systems that can accurately understand the universe and aid humanity in its pursuit of knowledge. The role involves designing and implementing complex data processing systems and tools for enhancing data quality and discoverability for pre-training and post-training across various modalities.
Responsibilities:
- Design and implement petabyte-scale, high-throughput data processing systems that involve both CPU- and GPU-based processing
- Design and implement tools for orchestrating complex data pipelines
- Design and implement innovative tools for improving data discoverability and data quality at scale for both pre-training and post-training across different modalities
- Build, run, and manage innovative data pipelines for creating high-quality training data