Top AI companies are asking for supervision. Here’s why Congress might not give it to them.

Por Brendan Bordelon — POLITICO – TOP Stories

OpenAI threw its weight Tuesday behind an increasingly popular proposal to address the dangers of artificial intelligence — requiring the top AI developers to embed independent watchdogs to ensure their products are safe.

The idea still faces obstacles, including complaints from one key lawmaker that the independent monitors would be too weak to impose meaningful changes.

Other critics, including venture capitalists, academics and President Donald Trump’s former AI czar, have expressed suspicion of financial and personnel ties between some monitoring organizations and the big AI labs. They allege that the proposed safety scheme would mainly serve to cement OpenAI and its top rivals atop the industry.

Yet the notion of placing third-party monitors or “assessors” to audit the world’s leading AI developers has gained momentum in recent days, amid rising warnings from inside the industry that the technology could trigger devastating cyberattacks, pandemics or other threats to humanity. The proposal offers one potential middle path between Trump’s opposition to AI guardrails and a push by progressive Sen. Bernie Sanders (I-Vt.) and Rep. Greg Casar (D-Texas) to ban the most powerful forms of the technology.

Governors and lawmakers in states such as California, Massachusetts and Utah are increasingly endorsing similar monitoring proposals. And top AI companies are already embracing the idea on a voluntary basis.

In an essay on Saturday that called for a coordinated slowdown of advanced AI development, Anthropic CEO Dario Amodei pledged that his company would provide “employee-like access to a team of embedded third-party evaluators.” OpenAI CEO Sam Altman made a similar promise hours later. And OpenAI chief global affairs officer Chris Lehane went a step further on Tuesday, saying his company supports a bipartisan House plan that would force OpenAI, Anthropic, and other AI giants to embed outside observers.

“We see it as a starting gun,” said Andrew Freedman, the co-founder and CEO of Fathom, a nonprofit advocacy group working to advance legislation in the states and on Capitol Hill that would require independent assessments of cutting-edge AI models.

Freedman said new pledges from OpenAI and Anthropic to embrace third-party monitoring represent “a final recognition that this is going to have to be a central pillar of governance,” highlighting growing support for the idea in statehouses and C-suites across the country.

The monitoring mandate is a key provision of the FRONTIER Act, legislation from Reps. Jay Obernolte (R-Calif.) and Lori Trahan (D-Mass.) that would create the first federal framework for AI safety.

Under the bill, AI developers that meet high thresholds for annual revenue and computing investments would have to allow “independent verification organizations” into their companies to make sure their models are developed safely. That term currently encompasses a stable of groups mostly based in the San Francisco Bay Area.

Hours after OpenAI endorsed the bipartisan proposal, it received pushback from Rep. Josh Gottheimer (D-N.J.), who co-chairs the Democrat-only House Commission on AI. He said Tuesday that top AI developers should instead have to submit their models for mandatory government vetting before releasing them to the public.

Tech industry enthusiasm for entrusting the vetting to private-sector monitors “should raise red flags, not confidence,” Gottheimer said.

‘It’s like a religious sect’

Gottheimer is hardly alone in his concerns. The question of who should play the oversight role, and the extent to which they could carry water for the AI giants, is one of the big questions facing the proposal.

Critics complain that most of the monitoring organizations are too close to Anthropic and other top AI companies, and that they contain concerning links to a Silicon Valley ideology known as effective altruism, which is often accused of being unusually fixated on an AI apocalypse.

“It’s like a religious sect,” said Benjamin Recht, a professor of computer science at the University of California at Berkeley. “You look at their background, they all come from the same place. And they come from the same place as Anthropic.”

Recht said mandatory AI audits would likely lead to “regulatory capture” by OpenAI, Anthropic and other leading AI firms. He worried that third-party auditors with ties to the companies could downplay mistakes made by humans — such as OpenAI’s failure to block its AI models from obtaining internet access during this summer’s Hugging Face hack — and instead blame the technology itself for security incidents.

An Anthropic spokesperson declined to comment on allegations of significant ties between the AI giant and the broader community of third-party AI evaluators, and would not say whether the company supports the bipartisan House proposal mandating the use of outside safety groups.

Lehane acknowledged on Tuesday that it is important for outside safety assessors to be seen as independent from his company and other top firms.

“It’s not like, the first time a company has had to do this,” the OpenAI official told POLITICO. “At one point, you had a whole process about which accounting firms were qualified to do financial and fiscal reviews. You see this in other industries.”

“We’re going to need to build up a body of who these folks are,” Lehane added. “And I do think it does need to be people who are broadly understood — again, for public confidence — to be independent and capable and competent.”

In recent days, support for the idea of embedding third-party evaluators within top AI companies has surged across the tech industry.

Last week, lobbyists for OpenAI belatedly backed a California bill establishing an accreditation and vetting process for outside AI auditing groups, announcing its support for the measure just as Democratic Gov. Gavin Newsom signed it into law. (Anthropic lobbyists endorsed the legislation in August.) In Massachusetts, lawmakers continue to debate a bill that would create the first statewide requirements for independent third-party AI assessments.

On Capitol Hill, the Obernolte-Trahan bill is getting a renewed look from lobbyists. One representative from a major tech trade association, granted anonymity to freely discuss internal talks, said Monday that their organization is taking a closer look at the bill’s monitoring proposal.

Also Monday, Utah Republican Gov. Spencer Cox urged Congress to pass legislation that requires independent evaluations of powerful AI models “throughout the development process.”

And Nvidia CEO Jensen Huang explicitly endorsed third-party AI safety evaluations — “that’s no different than financial control,” he said Monday, likening the process to financial auditors.

An AI staffer in Trahan’s office, granted anonymity to freely discuss internal deliberations, called it a “really strong signal that the leaders of these labs jumped on board with Dario’s embedded evaluators idea.”

“They’re moving in the right direction and they’re moving beyond words. I think that’s all helpful,” the staffer said. The person added that the next few weeks “will be critical to see sort of how those firm commitments translate to actual legislation that the entire industry would be regulated by.”

Who watches the watchers?

At the same time, powerful voices in the venture capitalist community continue to push back against the idea of third-party AI evaluators.

Jeremy Kudon, executive director of the American Innovators Network — a lobbying group for “little tech” funded by venture capital giants Andreessen Horowitz and Y Combinator — said the idea “leaves important questions like standards and the developing body of those standards unanswered all while being potentially developed and implemented without a public hearing or industry input.”

In a statement to POLITICO, Kudon worried that smaller AI companies and startups would struggle to navigate requirements to embed outside auditors.

Venture capitalist and former White House AI czar David Sacks took aim at Model Evaluation and Threat Research, an AI risk auditing nonprofit that Anthropic’s Amodei specifically named as a viable example of an independent evaluator.

“Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff,” Sacks told Amodei and Altman in an X post during the weekend. Sacks continues to have the ear of Trump, who on Monday accused AI industry leaders of propagating a “hoax” that their technology could harm humanity.

In an interview with POLITICO, METR President Chris Painter acknowledged that his organization contains substantial ties to researchers at Anthropic and OpenAI, as well as some links to the effective altruist ideology. He called it a natural, if unfortunate, outgrowth of hiring researchers who work at the cutting edge of AI development and alignment.

“If you’re trying to hire and build a team of people that is focused on informing the public about risks that might come from like AI autonomy and loss of control, and you’re doing that in a period, in an era where not much of the world is paying attention to that issue, you are going to inherently have this problem of hiring the people who are early to caring about that,” Painter said.

He stressed that METR highlights possible conflicts of interest in its reports, has turned down financing from prominent AI leaders and does not take money from Coefficient Giving, a funder of effective altruist causes backed by billionaire Facebook co-founder Dustin Moskovitz.

But just last week, Coefficient Giving announced it is setting aside $200 million in grants to build independent AI safety organizations. And many within industry are resigned to the influence that the group, and the effective altruism movement more broadly, will continue to have on third-party AI evaluators.

“They’re the only people putting money behind this stuff,” said one employee at a top AI company, granted anonymity to freely describe sensitive industry dynamics.

The Trahan staffer called the influence of AI industry researchers on ostensibly independent evaluators an “unavoidable problem,” given the small talent pools at play. But Freedman, the Fathom CEO, said he hopes AI auditing groups can shed some of their potential conflicts as the stable of outside auditors grows larger and more mainstream.

“We can’t let the perfect be the enemy of the good,” Freedman said. “This is an ecosystem in its nascency, and so there’s going to be things that we don’t want to exist in the next generation that exist right now.”

But Recht, the Berkeley professor, said AI auditing is always “toothless” and “enables regulatory capture.” He called on governments to instead fine AI companies or levy criminal penalties after cyberattacks, or to pass laws protecting open-source AI “so that any party can inspect what’s happening inside the labs.”

“There are solutions here that have a lot more teeth,” Recht said.

Fonte: POLITICO – TOP Stories