Principal Researcher, Benchmarks, Evaluation, and Standards

The Partnership on AI (PAI) is seeking a Principal Researcher to lead its technical work on benchmarking, evaluation, and standards. The role sets strategy, develops sociotechnical research and guidance, builds partnerships, and helps translate findings into implementable evaluations, standards, and recommendations.

Responsibilities:

• Develop PAI's point of view on the evaluation and benchmarking ecosystem, including assessing the infrastructure required for PAI to successfully develop and run its own evaluations and directing PAI’s strategy for integrating into the evaluation ecosystem, in collaboration with supervisor and executive team
• Advance the adoption of shared industry guidance through development of and integration into benchmarks, evaluations, and technical standards, in collaboration and partnership with relevant organizations in the auditing and certification space
• Offer technical leadership and shape the strategic direction of PAI’s focus areas (AI, Human Connection, and Well-Being, AI and Shared Prosperity, and Trustworthy AI Value Chains) in collaboration with those leading those focus areas and with the Chief Programs and Insights Officer
• Support technical efforts from other PAI teams, including Policy and International
• Work with PAI’s Partnerships team to structure work to support fundraising efforts
• Consider which questions across PAI can be addressed most effectively and urgently with sociotechnical solutions that PAI is well placed to develop, and design and build implementation pathways
• Work closely across PAI projects in order to ensure that guidance is technically feasible and implementable, e.g., as evaluations or standards, and supporting testing and sandboxing exercises related to PAI guidance where relevant
• Design approach for maintaining PAI’s benchmarking, evaluation, and standards work over time, including quality control for established partnerships
• Author/co-author peer-reviewed research papers, policy briefs, and accessible reports that translate PAI’s insights into actionable recommendations for industry, policymakers, and civil society, with an emphasis on bridging technical and social considerations
• Initiate and lead collaborative research partnerships with academic institutions, civil society organizations, and industry Partners to advance programmatic work
• Represent PAI at select conferences (e.g., NeurIPS) and other venues, presenting findings and methodological innovations
• Run point on key relationships with benchmarking, technical standards, and evaluation-oriented organizations and settings, including AISIs and assurance bodies
• Collaborate with Partnerships team to test approaches to earned-revenue models, including but not limited to licensing or content contributions to credentialing and training programs
• Translate rigorous findings into clear, compelling narratives for diverse audiences, including industry and civil society leaders, policymakers, and affected communities
• Support strategic communications efforts to amplify and increase the adoption of PAI's guidance
• Represent work to standards bodies, policymakers, media, funders, and the public, and connect open research questions to the resourcing that can answer them