In the high-stakes world of cybersecurity, the arrival of a new threat is often preceded by a whisper—a strange line of code or an anomalous file signature that doesn’t quite fit the established patterns. For Anton Cherepanov, a seasoned researcher at ESET, that whisper arrived in late August of last year. While monitoring VirusTotal, a collaborative platform where the world’s security community aggregates and analyzes suspicious files, Cherepanov flagged an upload that initially appeared benign but triggered his team’s custom heuristic sensors.
After hours of forensic dissection alongside his colleague Peter Strýček, the researchers realized they were looking at a paradigm shift. The file was a strain of ransomware, a category of malware designed to lock a victim’s data behind encryption until a payment is made. However, this specific sample, which they named "PromptLock," operated with a level of autonomy and sophistication never before seen in the wild. It didn’t just carry a static payload; it was integrated with Large Language Models (LLMs) to orchestrate its attack.
PromptLock was capable of using an LLM to generate malicious code on the fly, tailoring its behavior to the specific environment it had infected. It could autonomously map a local network to find high-value targets, exfiltrate sensitive data, and even draft personalized ransom notes based on the actual content of the files it had encrypted. Most disturbingly, the malware was polymorphic; because it relied on generative AI to produce its instructions, the code changed slightly with every execution, making it a "moving target" that traditional signature-based antivirus software struggled to identify.
While the discovery sent shockwaves through the industry, a surprising twist followed. Shortly after Cherepanov and Strýček published their findings, a research team from New York University (NYU) stepped forward to claim responsibility. PromptLock was not a product of a criminal syndicate, but rather a proof-of-concept designed to demonstrate that a fully automated, AI-driven ransomware campaign was technically feasible. The NYU researchers had successfully bridged the gap between theoretical AI threats and functional, autonomous digital weapons.
The Democratization of Cybercrime
The transition of AI from a productivity tool to a weapon of digital mass destruction represents a fundamental shift in the economics of cybercrime. Historically, launching a sophisticated cyberattack required a high degree of technical proficiency, significant time for reconnaissance, and the manual labor of writing and debugging exploits. Generative AI is rapidly eroding these barriers to entry.
Today, the same tools that help software engineers write cleaner code and identify bugs are being repurposed by "script kiddies" and organized crime groups alike. AI acts as a force multiplier, allowing less experienced attackers to orchestrate complex campaigns that would have previously been beyond their reach. Lorenzo Cavallaro, a professor of computer science at University College London, notes that the increased efficacy and frequency of these attacks is no longer a "what if" scenario but a "sheer reality."
However, the threat landscape is not solely defined by "super-malware." Many experts, including Marcus Hutchins—the researcher famous for neutralizing the 2017 WannaCry outbreak—argue that the public obsession with "AI superhackers" can be a distraction from more immediate, less flashy risks. While fully autonomous malware is a looming concern, the current "gold rush" for criminals lies in the automation of social engineering and fraud.
The New Era of Social Engineering
Before the advent of generative AI, phishing and spam were largely numbers games characterized by poor grammar, generic templates, and low success rates. AI has changed the math. According to a 2025 report from Microsoft, the company blocked over $4 billion in fraudulent transactions and scams in a single year, many of which were likely augmented by AI-generated content.
Research from Columbia University and the University of Chicago suggests that at least half of all spam emails are now produced using LLMs. More concerning is the rise of "spear-phishing"—highly targeted attacks that impersonate specific individuals. By April 2025, the frequency of targeted email attacks using LLMs had nearly doubled compared to the previous year.
The danger extends far beyond text. The maturation of deepfake technology has allowed criminals to impersonate the likeness and voice of trusted figures with terrifying accuracy. In one landmark case from 2024, a finance worker at the engineering firm Arup was deceived into transferring $25 million to a criminal account. The worker believed they were on a video conference with the company’s Chief Financial Officer and several other colleagues. In reality, every other participant on the call was a digitally rendered deepfake.
Henry Ajder, a leading expert in generative AI, warns that as long as these methods remain profitable, their usage will only accelerate. "If there’s money to be made and people continue to be fooled by it, they’ll continue to do it," Ajder says. The problem is exacerbated by the fact that creating a convincing deepfake now requires significantly less data and computing power than it did just twenty-four months ago.
Bypassing the Guardrails
The tech giants behind the most popular AI models, such as Google, OpenAI, and Anthropic, have invested heavily in "guardrails"—software limitations designed to prevent their models from generating malicious code or assisting in illegal acts. Yet, these defenses are proving to be porous.
Billy Leonard, tech leader of Google’s Threat Analysis Group, has documented a steady evolution in how "bad actors" interact with AI. In 2024, attackers were primarily using models like Gemini for mundane tasks: debugging scripts or drafting basic phishing lures. By 2025, they had moved toward using AI to create entirely new malware families.
One common tactic is "jailbreaking," where attackers use clever prompting to trick an AI into ignoring its safety protocols. In one instance, a China-linked threat actor successfully persuaded Google’s Gemini to identify vulnerabilities in a compromised system by pretending to be a participant in a "capture-the-flag" cybersecurity competition. By framing the request as a game or a research exercise, the attacker bypassed the model’s refusal to assist in a real-world hack.
Furthermore, the rise of open-source AI models provides a playground for criminals who want to operate without any oversight. Unlike closed-source models that are constantly monitored by their creators, open-source models can be downloaded, stripped of their ethical alignment layers, and fine-tuned specifically for malicious purposes. Ashley Jess, a senior intelligence analyst at Intel 471 and former tactical specialist at the U.S. Department of Justice, believes open-source models will become the preferred tool for the next generation of cybercriminals.
Agentic Orchestration: The Next Frontier
We are now entering the era of "agentic" AI—systems that can not only generate text or code but can also take actions and make decisions to achieve a specific goal. In November, the AI safety firm Anthropic reported that it had disrupted a large-scale espionage campaign orchestrated by a Chinese state-sponsored group. This campaign was notable because it used "Claude Code," an AI assistant, to automate roughly 90% of the intrusion process.
This was the first documented case of a sophisticated cyber-espionage operation executed without "substantial human intervention." While a human operator still selected the targets, the AI was responsible for identifying vulnerabilities and navigating the target’s internal systems.
Despite the alarming nature of this report, there are still significant technical hurdles preventing AI from becoming an unstoppable hacker. Anthropic noted that their model frequently "hallucinated," claiming to have obtained credentials that it hadn’t actually found. These errors require human attackers to constantly validate the AI’s work, which currently acts as a speed limit on the pace of autonomous attacks.
Gary McGraw, co-founder of the Berryville Institute of Machine Learning, remains skeptical of the "AI hype" in cybersecurity. He points out that many of the tools used in the Anthropic-documented attack—such as vulnerability scanners and automated exploit kits—have existed for two decades. "There’s nothing novel, creative, or interesting about this attack," McGraw argues, suggesting that AI is simply a new interface for old methods.
The Defensive Response: AI vs. AI
While the headlines often focus on the offensive capabilities of AI, the defensive side of the industry is undergoing an equally rapid transformation. The core strength of AI—its ability to process and find patterns in massive datasets—is a natural fit for cybersecurity defense.
Microsoft Security, for example, now processes over 100 trillion signals per day. No human team could ever hope to parse this much data, but AI systems can flag anomalies, correlate disparate events, and predict where an attacker might strike next in real-time. This "active defense" allows organizations to shut down compromised accounts or isolate infected servers before a human analyst even realizes there is a problem.
Transparency and information sharing have also become critical weapons. Organizations like MITRE and the Open Worldwide Application Security Project (OWASP) have launched initiatives to document how AI is being used in attacks. By creating a common language and database for AI-driven threats, the security community is attempting to build a collective immune system.
The Long-Term Outlook
The future of cybercrime will likely be defined by a perpetual arms race between increasingly autonomous offensive agents and increasingly sophisticated defensive filters. While we have yet to see a "cyber-Hiroshima"—a massive, clearly AI-driven attack that cripples global infrastructure—the groundwork for such an event is being laid.
The most extreme concern is the development of an AI model capable of discovering and exploiting "zero-day" vulnerabilities (security flaws unknown to the software’s creators) entirely on its own. Engin Kirda, a professor at Northeastern University, believes that wealthy nation-states, particularly China, are already investing billions into this capability. The cost of training and hosting such a model would be astronomical, likely keeping it out of the hands of common criminals, but placing it firmly within the arsenal of geopolitical adversaries.
For the average user and the typical corporation, the immediate future is one of increased vigilance. Generative AI has lowered the "floor" of cybercrime, meaning that the sheer volume of "good enough" attacks—phishing, deepfakes, and automated ransomware—will continue to rise.
As we navigate this new landscape, the foundational principles of digital hygiene remain more relevant than ever. Multi-factor authentication, regular software updates, and a healthy skepticism of digital communications are no longer just "best practices"—they are the essential armor in a world where the person on the other side of the screen might not be a person at all, but a silicon-based saboteur.