Hackers exploit Meta’s AI support bot, exposing flaws in automated security systems

Meta's efforts to automate customer support with artificial intelligence have come under scrutiny after hackers reportedly exploited the company's AI-powered support chatbot to take over several high-profile Instagram accounts. The incident exposed what security experts say is a fundamental risk of allowing AI systems to handle sensitive account recovery and security functions.

The breach allowed hackers to seize control of several prominent accounts, including the dormant Obama White House page, beauty retailer Sephora, and the account of a senior U.S. Space Force official. According to reports, the chatbot was persuaded to reset account credentials without independently verifying users' identities, effectively turning a trusted security tool into a major vulnerability.

More from Tech

The attack was carried out by unidentified hackers over the weekend and left affected users locked out of their accounts, triggering a wave of complaints on platforms such as X and Reddit. The incident was first reported by 404 Media on Monday and marks the latest setback for Meta as it expands the use of AI across its products and services.

Since the launch of ChatGPT in late 2022 sparked a global rush to deploy AI chatbots, cybercriminals have increasingly exploited prompt injection and manipulation attacks to compromise these systems. One widely cited example involved a Chevrolet dealership chatbot that was tricked into offering a Tahoe SUV for just $1 after an attacker manipulated its responses.

Experts say the Meta incident underscores a broader challenge facing AI-powered services and autonomous agents.

"It's not a Meta-specific issue. People are using these AI agents to do a lot of stuff. What we're actually seeing is unexpected problems that are coming up with the use of AI," said Engin Kirda, a professor in the Department of Electrical and Computer Engineering at Northeastern University.

Kirda added that the threat landscape is evolving alongside AI adoption. "In the past, people were targeted by scams. Now, we are seeing agents being targeted by scams," he said, referring to AI agents and autonomous digital assistants that are increasingly being entrusted with complex tasks ranging from customer support to online transactions.

The breach has renewed concerns about relying on AI systems for critical security functions such as password resets and account recovery. According to reports, some of the stolen account handles were subsequently listed for sale on the Telegram messaging platform.

Meta rolled out its AI support assistant globally across Facebook and Instagram earlier this year as part of a broader effort to integrate AI into customer support and other platform functions. The assistant was designed to handle tasks such as reporting scams, flagging impersonation accounts, addressing problematic content, and assisting users with password resets.

"The Meta AI support assistant is a major step in our work to deliver stronger support on our apps," the company had said at the time of launch.

The incident comes as Meta continues to invest heavily in artificial intelligence. The company has developed a series of large language models that power many of its AI features, while CEO Mark Zuckerberg has repeatedly outlined ambitions to build AI systems capable of achieving "superintelligence" — a term used to describe AI that surpasses human capabilities across a wide range of cognitive tasks.

The latest breach is likely to intensify scrutiny of Meta's efforts to automate increasingly sensitive functions. Critics argue that incidents such as this highlight the risks of entrusting AI systems with security-critical responsibilities before adequate safeguards and verification mechanisms are in place.

(With Inputs from Reuters)