I’ve said from the beginning that the biggest risk with AI isn’t that it writes bad code. It’s that people are starting to trust it to make decisions it has no business making.
New research drives that point home.
Researchers have uncovered a new attack called HalluSquatting, and it exploits one of AI’s most well-known weaknesses: it would rather make something up than admit it doesn’t know the answer.
That behavior, which we’ve all come to know as hallucination, is no longer just an annoyance. It can now be turned into a weapon.
Here’s how it works.
Developers increasingly rely on AI coding assistants like GitHub Copilot, Cursor, Gemini CLI, Windsurf, Cline, and others to clone repositories, install packages, and automate development tasks. If the AI doesn’t know where a newer project lives, it often invents a repository location that looks believable.
The scary part is those made-up locations aren’t random.
Researchers found they follow predictable patterns.
That means an attacker can register one of those fake repository names before anyone else does, upload malicious code, and simply wait. Eventually an AI assistant hallucinates that exact location, downloads the attacker’s repository, and in many cases executes whatever instructions it finds there.
Think about that for a second.
The attacker isn’t tricking the developer.
The AI is tricking itself.
And because modern coding assistants often have access to command shells, terminals, package managers, and development environments, they can end up installing malware, opening reverse shells, or executing scripts with very little user involvement.
At that point, you’ve effectively turned an AI assistant into a malware delivery platform.
What makes HalluSquatting different from traditional prompt injection attacks is scale.
Most prompt injection attacks require the attacker to target individual users by hiding malicious instructions inside emails, documents, source code, or web pages. Every victim has to encounter that content individually.
HalluSquatting flips the model.
Instead of pushing malicious content toward thousands of users, the attacker registers a malicious repository once and waits for AI agents to come looking for it. As adoption of AI coding assistants continues to grow, that attack scales almost automatically.
Researchers say the technique could be used to build botnets, deploy ransomware, install cryptocurrency miners, or create persistent backdoors across thousands of systems.
The underlying problem isn’t really the attack itself.
It’s that large language models still don’t know how to say three simple words.
“I don’t know.”
Humans verify information when they’re unsure.
LLMs tend to generate an answer that sounds plausible, even when it’s completely fabricated.
That’s manageable when the output is a paragraph or an email draft.
It’s a much bigger problem when the AI has permission to download software and execute commands on your machine.
One of the more interesting findings from the research was how often this happens with newer projects. Older repositories that existed before the models were trained were usually resolved correctly. But newer repositories, especially those created after the model’s knowledge cutoff, were frequently hallucinated. In some cases, the major language models generated incorrect repository locations more than 90 percent of the time.
That’s a staggering number.
It also explains why this attack is so practical.
The attacker doesn’t need to guess what the AI might invent. The models are surprisingly consistent, which makes those fake repository names relatively easy to predict and register ahead of time.
In a lot of ways, this reminds me of the early days of typo-squatting, where attackers registered package names that looked almost identical to legitimate software. The difference is that humans aren’t making the typo anymore.
The AI is.
That shifts the security conversation in a completely different direction.
We’re moving toward AI agents that can write code, install dependencies, deploy infrastructure, and administer systems with very little oversight. Every new permission we hand these tools increases the damage they can do when they inevitably make a bad decision.
That doesn’t mean AI coding assistants aren’t useful.
They absolutely are.
But they still need supervision.
The safest approach is to treat AI the same way you’d treat a brand-new junior developer. Let it do the repetitive work. Let it save time. Just don’t let it make security decisions on its own, and definitely don’t let it blindly download and execute code from the internet without verifying where it came from.
The technology keeps getting better.
The trust model hasn’t.
