METR is a nonprofit research organization focused on understanding AI capabilities and risks. They are seeking a researcher to contribute to various streams including capabilities, monitorability, and alignment, with responsibilities spanning research science, execution, and management.
Responsibilities:
- Accurately measuring frontier model performance on threat-relevant tasks (autonomy, AI R&D automation, etc.) and predicting future capabilities
- Develop and maintain benchmarks, diverse evidence-gathering methods, and metrics to track capability trends and anticipate the thresholds that matter most for safety
- Understanding how well frontier models can take subversive or unwanted actions despite various monitoring or control protocols
- Build the research infrastructure – novel metrics, control evaluations, elicitation methods – needed to improve the world's understanding of how effectively current and future models can circumvent oversight
- Determine whether or not a model that is capable of causing catastrophic harm (in its actual deployment setting) would be likely to actually do so in a given high-stakes deployment setting
- Develop the science of propensity evaluations and examine when we might expect high-stakes catastrophic misalignment
- Set a compelling and coherent research agenda for a team - translate high-level goals into tractable projects that serve METR's mission
- Hire and develop researchers; invest in the growth of team members
- Evaluate the work of researchers across a range of subproblems, give useful feedback, and catch things going wrong early
- Communicate clearly with leadership and stakeholders - surface the right information, flag risks proactively, and represent the team's work effectively