Member of Technical Staff, Research

METR is a nonprofit research organization focused on understanding AI capabilities and risks. They are seeking a researcher to contribute to various streams including capabilities, monitorability, and alignment, with responsibilities spanning research science, execution, and management.

Responsibilities:

• Accurately measuring frontier model performance on threat-relevant tasks (autonomy, AI R&D automation, etc.) and predicting future capabilities
• Develop and maintain benchmarks, diverse evidence-gathering methods, and metrics to track capability trends and anticipate the thresholds that matter most for safety
• Understanding how well frontier models can take subversive or unwanted actions despite various monitoring or control protocols
• Build the research infrastructure – novel metrics, control evaluations, elicitation methods – needed to improve the world's understanding of how effectively current and future models can circumvent oversight
• Determine whether or not a model that is capable of causing catastrophic harm (in its actual deployment setting) would be likely to actually do so in a given high-stakes deployment setting
• Develop the science of propensity evaluations and examine when we might expect high-stakes catastrophic misalignment
• Set a compelling and coherent research agenda for a team - translate high-level goals into tractable projects that serve METR's mission
• Hire and develop researchers; invest in the growth of team members
• Evaluate the work of researchers across a range of subproblems, give useful feedback, and catch things going wrong early
• Communicate clearly with leadership and stakeholders - surface the right information, flag risks proactively, and represent the team's work effectively