From FraudGPT to deepfakes: AI is now running cybercrime at scale — and only AI can stop it

"Can machines think?" — a seminal question that has shaped the profound tenets of computer science over the decades. When Alan Turing, often regarded as the father of theoretical computer science, raised it, he could scarcely have imagined what artificial intelligence would become. Yet the question remains perennial, and its weight has only grown.

Every innovation carries the possibility of misuse. AI's steady inclusion in every facet of life has introduced a myriad of threats that users are often unaware of. The recent case involving Phoenix Ikner — who allegedly held extensive conversations with ChatGPT linked to plans for a school shooting — has highlighted the grave dangers that even a single AI failure can pose.

More from Tech

But this is only one facet of a much larger picture. AI is now being actively deployed to develop new methods of cybercrime. Models such as Mythos have sparked concerns over the possibility of cyberattacks evolving from human-led operations to autonomous, AI-driven exploitation at scale.

Traditionally, hackers required time, skill, and coordination. AI can streamline their methods and enable them to operate on a scale that was previously unimaginable.

As Sanjay Katkar, joint managing drector at Quick Heal, explains: "The coming phase of cyberattacks is expected to become increasingly autonomous, adaptive, and powered by intelligent automation, with AI-driven phishing, impersonation campaigns, intelligent botnets, automated vulnerability exploitation, and advanced persistent threats (APTs) expected to rise significantly."

"What we are witnessing is not the emergence of entirely new attack vectors, but the amplification of existing ones at machine speed and scale, making threats more behaviour-driven and considerably harder to detect through traditional controls."

But the picture is not one-sided.

"At the same time, AI is also becoming a critical component of modern defence strategies. Organisations are increasingly leveraging AI for security to identify anomalies, combat AI-assisted phishing, detect malicious botnet activity, mitigate DDoS attacks, and improve responses against evolving attack campaigns in real time," says Katkar

Mitre Atlas and the new exploit frontier

Mitre Atlas (Adversarial Threat Landscape for Artificial-Intelligence Systems) conducted a rapid investigation into Open Claw, analysing critical incidents identified by the AI security community and mapping associated threats to Atlas tactics.

The investigation aimed to show how AI-first ecosystems have introduced entirely new exploit execution paths that traditional security models do not cover.

It found that tactics such as prompt injection, model manipulation, adversarial attacks, data poisoning, and model theft represent emerging threats that security systems must now account for.

Another expert, Varun Grover, business unit head at mFilterIt, says, "Cybercriminals are using AI to industrialise deception at scale. From highly realistic websites and AI-generated influencers to deepfake endorsements, automated chat support, and fake subscription campaigns, fraudulent experiences are now designed to closely mimic trusted platforms like ChatGPT or Gemini."

Grover says, "The most alarming shift is hyper-personalisation. AI can analyse browsing habits, interests, purchase behaviour, and online engagement to create scams that feel highly relevant and credible to individual users, significantly lowering suspicion levels."

Key warning signs for users include unrealistic discounts, "free trial" traps asking for card details, hidden auto-renewals, fake influencer promotions, and suspicious payment flows, he says.

A new level of preparedness

Responding to these emerging threats, Indian Finance Minister Nirmala Sitharaman and IT Minister Ashwini Vaishnaw recently chaired a high-level meeting with banks and regulators. Senior officials from the Reserve Bank of India, National Payments Corporation of India (NPCI), and the Indian Computer Emergency Response Team (Cert)  were in attendance.

Sitharaman emphasised that the nature of threats emerging from new AI models is unprecedented and requires greater preparedness and coordination across financial institutions. The Indian Banks' Association (IBA) was tasked with creating a coordinated response mechanism for AI-related threats.

Identity theft and digital fraud

In 2025, America's Identity Theft Resource Center (ITRC) — which has tracked digital identity fraud since 2005 — recorded the highest number of data compromises in the United States since it began. AI has become a powerful force in cybercrime. Michael Bruemmer, vice president of Consumer Protection, was quoted by a Bloomberg report as saying that 40 per cent of the 5,000 data breaches experienced by consumer credit agency Experian last year were reportedly powered by AI.

The report also noted that AI subagents can actively scan the dark web for vulnerable Social Security numbers and personal information within seconds.

AI's integration into cyber fraud means criminals can now launch simultaneous attacks across multiple banks while impersonating different identities and fill out fraudulent loan applications at scale.

Beyond traditional methods of stealing Social Security data and identity credentials, one of the most alarming emerging trends is synthetic fraud — combining real and fake data to create highly realistic yet fictitious identities used to open bank accounts or secure lines of credit.

LLM-based tools such as FraudGPT have heightened these concerns further; such programs can reportedly test hundreds of thousands of Social Security numbers within minutes.

It is not all bleak, however. There are several guardrails that consumers can establish to protect themselves. The Bloomberg report noted that multi-factor authentication and password vaults secured with biometric passkeys are critical safeguards, as is using VPNs on public Wi-Fi networks.

Conducting quick checks on suspicious links and consulting cybersecurity forums to identify recurring scam patterns can also help users recognise attempts to steal sensitive information.

In the face of increasingly intelligent cyber threats, organisations can no longer afford to rely solely on reactive security measures. Building strong cyber resilience requires a proactive and adaptive approach that combines advanced AI-driven monitoring with disciplined cybersecurity fundamentals.

By prioritising real-time threat detection, rapid response, regular system updates, strict access management, and employee preparedness, businesses can stay ahead of evolving attacks.

The future of cybersecurity will depend not only on technology but on an organisation's ability to foster continuous vigilance. The question Alan Turing once raised — "Can machines think?" — AI has, in many ways, already answered. What remains urgent is ensuring that human judgement stays at the centre of how that thinking is directed.