Research Lead - Pre-training Safety

FAR.AI is a non-profit AI research institute working to ensure advanced AI is safe and beneficial for everyone. The Research Lead will develop and lead pre-training safety research focused on capability control, including data filtering, gradient routing, unlearning, and synthetic data, while building a technical team and scaling experiments on frontier models. The role also involves partnering with red teams, mentoring researchers, publishing findings, and driving adoption of successful methods.

Responsibilities:

• Articulate a research agenda with a clear theory of change for mitigating catastrophic risks from human-level or superhuman AI systems, and/or vastly increasing the upside of such systems
• Grow and lead a team of technical staff in pursuit of this agenda, either directly or in partnership with an engineering co-lead
• Lead novel research projects where there may be unclear markers of progress or success
• Share your research findings through written content (e.g. academic publications, blog posts) and presentations (e.g. ML conferences, policymaker briefings) to drive adoption and change
• Mentor and coach junior team members in research skills and ML engineering
• Contribute to the FAR.AI intellectual environment and research culture, for example by giving feedback on early-stage proposals
• Build a research field around your agenda through FAR.AI's grantmaking and events, and connect it to real-world deployments through our independent testing and government advising
• You'll build and lead the team, set its research direction, mentor Members of Technical Staff to scale your vision, and remain hands-on enough to write code and run experiments yourself
• Validate approaches like pre-training data filtering at scale, drive adoption of successful methods, and explore techniques such as gradient routing and unlearning
• Direct this work, partner with our red team to stress-test the resulting models, and analyze how well the methods scale to frontier systems