Anthropic co-founder joins Pope Leo to warn against leaving AI in Big Tech’s hands

The Vatican became an unlikely stage for one of the tech industry’s most urgent debates on Monday, as Anthropic co-founder Chris Olah warned that artificial intelligence must not be governed solely by Silicon Valley companies.

Appearing alongside Pope Leo during the presentation of the pontiff’s first encyclical on AI, Olah said the technology’s growing influence over economies, labour and society demanded broader oversight from governments, religious institutions and civil society groups.

The event underscored the Catholic Church’s attempt to establish itself as a moral voice in the global AI debate, at a time when concerns around automation, surveillance and opaque algorithms are intensifying worldwide.

‘A scary moment’ for society

Speaking before an audience at the Vatican, Olah described the current pace of AI development as both transformative and deeply unsettling.

More from Tech

“I think this is a scary moment. Things are moving fast. It’s a really powerful technology,” he told Reuters after the event.

He cautioned that advanced AI systems could eventually displace workers on a massive scale, reshaping industries faster than societies are prepared to handle. “There’s a real possibility” that AI could replace human labour “at very large scale”, he said, warning that helping those affected would become “a moral imperative of historic proportions”.

While many technology companies argue AI will ultimately create new forms of employment, Olah suggested the transition could still produce severe disruption and inequality if left unmanaged.

He identified three pressing challenges requiring urgent global attention: the threat of widespread job displacement, unequal access to AI’s benefits, and the growing difficulty of understanding how increasingly complex systems behave.

“AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?” he asked.

Olah also acknowledged that even companies building frontier AI systems are not immune to commercial and political pressures.

“Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing,” he said, arguing that outside scrutiny had become essential as companies race to develop more powerful systems.

Why the Vatican invited Anthropic

The Vatican’s decision to feature Olah prominently reflected his longstanding focus on AI safety and ethics. Asked why he appeared to be the only representative from a major AI company invited to the event, Olah pointed to his years of work examining the societal implications of the technology.

He said he had spent much of his career trying to make AI systems safer and had engaged with followers of more than 15 religions on ethical questions surrounding the technology.

Anthropic itself was founded in 2021 by former OpenAI employees, including Olah, who reportedly became concerned that the company behind ChatGPT was moving too quickly without sufficient safety testing. The startup later emerged as one of OpenAI’s biggest rivals through its Claude AI models.

The company has also publicly disagreed with aspects of President Donald Trump’s AI policies, particularly over safeguards limiting the military use of AI systems, including autonomous weapons targeting and domestic surveillance.

For the Vatican, the partnership highlighted a growing belief that the future of AI is not simply a technical question but a profoundly human one.

“There’s a risk that things could go badly,” Olah said. “And it’s incumbent on all of us to push this in a good direction.”