AI safety watchdogs face conflict of interest concerns as staff rotate between evaluators and companies

The people grading AI safety homework might also be the ones who wrote it. That’s the uncomfortable implication of growing scrutiny over the revolving door between AI safety evaluation organizations and the very companies they’re supposed to oversee. Proposals from Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman to embed third-party evaluators directly inside their labs have turned a simmering concern into a full boil. The web of connections At the center of the controversy sits METR, or Model Evaluation and Threat Research, one of the most prominent organizations tasked with assessing whether frontier AI models are safe enough for deployment. METR’s staff includes people who previously worked at OpenAI and Anthropic, and the overlap doesn’t stop at resumes. The funding picture is where things get especially tangled. Dustin Moskovitz, the Facebook co-founder, holds equity in Anthropic. His philanthropic foundations channel money through Effective Altruism networks that have provided significant funding to evaluator organizations like METR and Redwood Research. The result is an ecosystem where the money flowing to safety watchdogs can be traced back, sometimes through only a ho...