DEV Community

Cover image for 1 in 5 Packages Your AI Suggests Don't Exist. Attackers Know Which Ones.
James Anderson
James Anderson

Posted on AI-assisted

1 in 5 Packages Your AI Suggests Don't Exist. Attackers Know Which Ones.

Coined in 2025 by Python's Seth Larson

You ask your AI assistant how to do something. It gives you clean, confident code, with an install line at the top:

pip install aws-helper-sdk
Enter fullscreen mode Exit fullscreen mode

You run it. The build works. You move on.

Except aws-helper-sdk never existed. The model made it up. And last month, someone registered that exact name on PyPI — with malware inside — because they knew the model would suggest it.

That's slopsquatting, and it's the supply-chain attack built specifically for the AI coding era. The name was coined in 2025 by Seth Larson of the Python Software Foundation [1], and unlike most security scares, this one is measured, documented, and already in the wild. Let me walk through how it works, the numbers that make it real, and — the part you actually came for — how to not get caught by it.

Typosquatting needed your mistake. This one doesn't.

You already know typosquatting. An attacker publishes a malicious package called expres, betting that someone, someday, fat-fingers pip install express. It works occasionally, but it depends on a human error, and most people spell express correctly.

Slopsquatting flips the burden of the mistake. You don't have to slip. The AI makes the mistake for you — reliably, confidently, in code that looks completely correct — and the attacker registered the hallucinated name in advance. You did everything right. You just trusted the tool, and the tool invented a dependency that a stranger was already squatting.

That's the whole shift: the vulnerability moved from your carelessness to your trust.

The numbers that make this real

This isn't a thought experiment. The foundational study — "We Have a Package for You!", presented at USENIX Security 2025 — generated 576,000 code samples across 16 different LLMs and checked every package the models recommended [2].

The headline result: 19.7% of all recommended packages were hallucinated — nearly one in five. Across those hallucinations, the researchers logged 205,474 unique non-existent package names [2]. That's not a rounding error; that's a vast, ready-made attack surface.

The rate isn't uniform. Open-source models were worse (up to ~22%); commercial models did better, with GPT-4 Turbo the best performer at 3.59% [3]. And a 2026 re-evaluation of newer frontier models found the range had narrowed to roughly 4.6%–6.1% — better, but nowhere near zero [4]. The problem is shrinking, not gone.

So somewhere between 1-in-20 and 1-in-5 of the package names your AI hands you may point at something that doesn't exist. On its own, that's just an annoyance — a failed install. What turns it into an attack is the next fact.

The part that turns a bug into a weapon: it's predictable

Here's the insight that makes slopsquatting genuinely dangerous, and it's worth sitting with.

If hallucinations were random — a different made-up name every time — attackers couldn't do much with them. You can't register 205,000 names and hope someone hits the exact one you own. The noise would protect you.

But hallucinations aren't random. When the researchers took 500 prompts that had produced a fake package and re-ran each one ten more times, 43% of the hallucinated names came back on every single run [2]. Same prompt, same model, same invented package, over and over.

It gets sharper. A 2026 cross-model study found 127 package names that five different frontier models all invented identically [4]. Different tools, same phantom dependencies.

That predictability is the exploit. A systematic, repeatable model behavior becomes a namespace an attacker can register in advance [5]. They don't guess — they run the popular models against the popular prompts, harvest the hallucinations that reliably recur, register those exact names with malicious code, and wait. The model does the targeting for them, for free, every time a new developer asks a similar question.

Why your existing defenses don't catch it

You might think your supply-chain tooling has this covered. Mostly, it doesn't — because it was built for the last attack.

Typosquatting detection works on string similarity: how many single-character edits turn expres into express? That's a sensible defense against typos. But slopsquatted names aren't typos. The same study found that, by edit distance, only about 13% of hallucinated names were simple typos of real packages — nearly half were wildly dissimilar to anything that exists [6]. A name like aws-helper-sdk or fastapi-middleware [5] doesn't resemble a real package closely enough to trip a similarity check. It just sounds plausible — which is exactly what a language model is good at generating, and exactly what makes it slip past a human reviewer who isn't familiar with the specific ecosystem.

The defense built for human error is blind to machine error. That gap is the opening.

This is already happening

Slopsquatting isn't a projection. Malicious packages registered on hallucinated names are live in public registries right now. Security researchers documented one slopsquatted package that was still recording roughly 233 weekly downloads as of February 2026 — even after npm had placed a security hold on it [3]. People (and their agents) were still pulling it.

The propagation vector is worse than individual installs, too. When a hallucinated install command lands in documentation — a README, a tutorial, an official-looking guide — it spreads to everyone who copies it. There are documented cases of AI-recommended install commands for non-existent packages making it into trusted institutional docs [3]. One hallucination, copied into one popular guide, becomes thousands of installs.

And there's a structural reason the attack surface is so large: roughly 90% of the open-source ecosystem is dormant — millions of abandoned forks and experiments nobody looks at [5]. Humans stay on the well-trodden paths. Models sample the entire internet, including the forgotten corners, which is part of why they surface names that sound real but aren't.

How to actually defend against it

Here's the part you can act on today. There's no single magic fix — the honest defense is a few layered, boring controls plus one mindset change. The mindset change is the important one:

Treat any package name an LLM gives you as untrusted input, not a verified fact. A package name emitted by a model is a claim, not a citation. Every claim gets checked before it becomes a dependency. If you internalize only one thing, internalize that.

Concretely:

  • Verify before you install. Before running an AI-suggested install, actually look the package up. Does it exist? How old is it? How many downloads? Does it have a real repo, real maintainers, real history? A package that "sounds official" but was published three weeks ago with 200 downloads is a screaming red flag — that's what a freshly-registered slopsquat looks like.
  • Pin and hash your dependencies. Use lockfiles (package-lock.json, poetry.lock), pin versions, and where you can, require hashes (pip install --require-hashes). Nothing should enter the build that you didn't deliberately vet.
  • Put a gate between the model and the registry. For teams: a private registry, an allow-list, or a dependency firewall so only approved packages resolve. The AI can suggest whatever it wants; only vetted names actually install.
  • Scan new dependencies in CI, and flag the new ones for human eyes. A dependency scanner plus a rule that any newly added package gets a human review catches the slopsquat before it merges.
  • Scan your own docs and READMEs. Hallucinated install commands propagate by copy-paste. Check your documentation for package-manager commands the same way you'd check code.
  • This one is critical for agents: if you have an autonomous coding agent that runs its own install commands, it will pull the malware with zero human in the loop. An agent that can install is an agent that can be slopsquatted at machine speed. Gate its installs behind verification or approval — do not let it self-install unvetted packages.(Some agent platforms are starting to build these install gates in — Xenition, which I work on, is one — but the principle matters more than any tool.)

None of these is exciting. All of them together turn "the AI said so, so I ran it" into "the AI suggested it, so I checked it." That's the whole defense.

The takeaway

Typosquatting punished carelessness — you had to make the mistake. Slopsquatting punishes trust — specifically, the quiet assumption that AI-generated code is basically correct, so the install line at the top is basically safe.

It isn't. About one in five of those lines (fewer on newer models, but never zero) points at nothing — and the ones that point at nothing point there predictably, which is exactly what lets an attacker be standing at the other end when you arrive.

So the rule is simple, and it costs you ten seconds: a package name from an AI is a claim to verify, not a fact to run. Check that it exists, that it's real, that it's the one you meant — every single time. Because the one time you don't bother might be the exact name someone registered last month, hoping you wouldn't.


Honest question: have you ever run an AI-suggested install command without first checking the package actually existed? Be honest — I have, more than once, before I knew this was a thing. What does your verify-before-install workflow look like now, and has anyone here actually caught a slopsquatted package in the wild?


Sources & further reading: [1] The term "slopsquatting" was coined by Seth Larson (Python Software Foundation) and popularized by Andrew Nesbitt, 2025. [2] Spracklen et al., "We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code-Generating LLMs," USENIX Security Symposium 2025 (576,000 samples across 16 LLMs; 19.7% hallucination rate; 205,474 unique fake names; 43% recurrence on rerun). [3] Reporting and analysis from Socket, the Cloud Security Alliance research note on slopsquatting (April 2026), and industry writeups (GPT-4 Turbo 3.59%; the ~233 weekly-downloads case as of Feb 2026; documentation-propagation cases). [4] 2026 frontier-model re-evaluation ("The Range Shrinks, the Threat Remains"): per-model rates ~4.6%–6.1%; 127 package names hallucinated identically across five models. [5] Snyk / industry commentary on predictability as the exploitable property, example hallucinated names, and open-source dormancy (~90%). [6] USENIX 2025 study's Levenshtein-distance analysis: ~13% of hallucinated names were simple typos; ~38% were "conflations" merging two real names. Figures come from academic and vendor research across 2025–2026; treat specific numbers as reported-as-of-writing and follow the primary sources — especially the USENIX paper — for methodology and the latest data.

Top comments (35)

Collapse
 
xxxn3m3s1sxxx profile image
xxxn3m3s1sxxx • • Edited

The "pin and hash" advice has a hole in it that only shows up once agents are the ones running the install: the lockfile is written by the same process that hallucinated the name. A pin you generated yourself is a self-attestation, not a check — the agent adds the package, resolves it, commits the lockfile, and the hash now faithfully records a bad decision. Nothing in the diff looks wrong either, because a lockfile entry is a wall of checksums no reviewer actually reads.

So the useful property isn't "is it pinned," it's "who was allowed to change the pin." What's worked for us is treating the dependency set as a reviewed artifact rather than a build byproduct: adding a name is a proposed change that a different process validates against the registry before it lands, and the agent that suggested the name can't be the one that approves its own resolution. Age-gating and allowlists both slot in naturally there, because that's the chokepoint where the request exists before it's already true.

The other half is that verification has to re-run outside the agent's write path — CI reading the lockfile from the repo is fine, CI trusting whatever the agent just reported is the same trust problem one layer up.

Collapse
 
slabb profile image
Sam LABBE •

The two-writer rule is the fix; the part that makes it survive an incident is turning the writers' outputs into evidence, not just separating them. Proposal, approval, and the CI re-resolution as three events from three writers, correlated by the resolution they refer to — when they disagree after the fact, you have a sequence to read, not an opinion to negotiate. Because the recursion in your last line keeps going otherwise: the CI that re-runs the check is itself a process whose report can be wrong, one layer up each time. What ends the regress is sealing each claim at write time — the chain doesn't vouch for truth, it vouches for who claimed what and when, which is what makes a silent bypass legible as "approval with no matching proposal."

Collapse
 
james_anderson_h profile image
James Anderson •

Turning outputs into evidence is what makes the two-writer rule survive the incident — proposal, approval, CI re-resolution as three correlated events gives you a sequence to read, not an opinion to negotiate. And you've ended the regress: verification can't be what you trust, because the verifier's report can be wrong one layer up, forever. Sealing each claim at write time vouches for who claimed what and when, not truth — so a silent bypass shows up as a hole in the sequence, not a judgment call.

Collapse
 
james_anderson_h profile image
James Anderson •

The sharpest hole anyone's found: when the agent runs the install, the lockfile is written by the same process that hallucinated the name, so the pin becomes self-attestation — the hash faithfully records a bad decision, and nobody reads a wall of checksums. Self-grading, one layer down. The fix is "who's allowed to change the pin": the dependency set as a reviewed artifact, validated by a different process before it lands. Verification must re-run outside the agent's write path.

Collapse
 
normalnorma profile image
Norma •

Great breakdown. The shift you describe, from the vulnerability being your carelessness to being your trust, is what makes slopsquatting different from typosquatting. Similarity-based detection is built around human typos, so it has nothing to say about a plausible-sounding name like aws-helper-sdk that a model invented.

The predictability finding is the part that turns this from an annoyance into an attack. If 43% of hallucinated names recur on every rerun, an attacker can harvest them systematically and register them in advance. The cross-model result, where five frontier models independently invented the same 127 names, makes that worse, because switching assistants doesn't protect you.

Your point about agents is the one I'd stress most. A human at least has a moment where they might glance at the package name. An autonomous agent that runs its own installs removes that moment entirely, so gating installs behind verification or approval matters more than any of the other controls.

A few additions to the defense list:

  • Check package age and publish date, not just existence. A package that was first published a few weeks ago and matches a name a model just suggested is a strong signal. Some tooling can flag "newly registered and matches an LLM suggestion" automatically.
  • Use install-time restrictions where the ecosystem allows it. Disabling install scripts by default (e.g. npm install --ignore-scripts) limits what a malicious package can do on arrival, even if it slips through.
  • Prefer name verification against the official docs. If an assistant suggests a package for a well-known library, confirm the name on that library's own documentation or repo rather than asking the model again, since it will often repeat the same hallucination.
  • Reserve your own likely names. For maintainers, registering obvious variants of your project's name (yourproject-sdk, yourproject-cli) closes off the cheapest squatting targets.

One question: do you know whether the recurrence rate differs between prompts that mention a specific framework and generic "how do I do X" prompts? My guess is that the vague, task-shaped prompts produce the more reliably exploitable names, since there's no real package to anchor on, but I haven't seen data either way.

The ten-second rule at the end, treating a package name from an AI as a claim to verify rather than a fact to run, is easy to remember and cheap to adopt. Thanks for the clear write-up and the sourcing.

Collapse
 
james_anderson_h profile image
James Anderson •

Those four additions strengthen the defense list materially, and --ignore-scripts is the one I most regret leaving out — it bounds what a malicious package can do on arrival, which is defense-in-depth even for the squat that slips the net. Reserving your own likely names is the cheap maintainer-side move nobody thinks of until it's too late. On your question: I don't have hard data splitting recurrence by prompt type, but your hypothesis matches the mechanism exactly — vague task-shaped prompts have no real package to anchor on, so the model fills the gap with its most probable invention, which is precisely the stable, recurring, registerable kind. Framework-specific prompts anchor on something real and fail toward the correct name more often. If that holds, the most exploitable hallucinations come from exactly the beginner-style "how do I do X" queries — which is the worst possible population to be exposed. Worth someone measuring directly.

Collapse
 
pepapepa profile image
pepapepa •

I don't like something around this user. trouble. bad mojo vibes.

Collapse
 
slabb profile image
Sam LABBE •

This is the "see you there" arriving, then — the buildable half, wearing a supply-chain badge.

Honest answer first: yes, more than once — and almost always in a throwaway environment, which is exactly the rationalization this attack is built around. "The build worked, I moved on" is the whole story.

The thing I'd push one step further: the predictability you flag cuts both ways. If 43% of hallucinated names recur on every rerun, then your model's hallucination set is finite and enumerable. Run your real prompt corpus, collect every package name it invents, and that list becomes regression fixtures and a seed denylist for your registry gate. Attackers harvest your model's hallucinations from the outside; the cheap defense is to harvest them from the inside, from your own traffic, first.

On "gate its installs" — a gate that leaves no record is a habit, not a control. After an incident, "we had a verification step" is a policy statement; what an auditor (or an insurer) needs is the sequence itself: which model emitted the name (a claim), what the registry check returned (a verification), what approved or refused the install (a decision) — three separate events, reconcilable after the fact, the way you'd reconcile a payment decision against a provider response. That's the pattern behind the flight-recorder journal I keep mentioning: an agent that can install is an agent that must journal.

And your docs point might be the sleeper: agents increasingly write the docs too, so the verification has to attach to the artifact and re-run in CI — not just to the moment someone pastes the line.

Nothing caught in the wild on my side yet. But the early-warning metric is now obvious: the fake name that comes back on every rerun is the one to watch.

Collapse
 
james_anderson_h profile image
James Anderson •

The inversion is the sharpest thing added: if 43% of hallucinations recur, the model's hallucination set is finite and enumerable — so run your own prompt corpus, harvest what it invents, and that list becomes regression fixtures and a seed denylist. Attackers harvest your model's hallucinations from outside; the cheap defense is beating them to it from the inside. That flips a scary stat into a build step.

And you're right that a gate with no record is a habit, not a control — three reconcilable events (claim, verification, decision) is the difference between "we had a step" and evidence. An agent that can install is an agent that must journal. See you there, indeed.

Collapse
 
slabb profile image
Sam LABBE •

Twice now, then — "see you there" has closed both of these. So, the address: the journal's reconciliation layer, where claim, verification and decision become typed, replayable events, is sitting open as an issue that needs skeptics more than applause. You know where it lives. Bring the attempted breaks.

Thread Thread
 
james_anderson_h profile image
James Anderson •

Noted the address — I'll come with breaks, not applause.

Collapse
 
anciwasim profile image
Wasim Sheikh •

Long-lived trust is how a generated package name turns into a supply-chain problem after the human has left the chat. We tie dependency approval to the task, verify the package and owner, and hard-stop when that envelope expires. “The model suggested it” is not a permission model.

Collapse
 
james_anderson_h profile image
James Anderson •

"The model suggested it is not a permission model" — that's the whole fix in a sentence. Tying approval to the task and hard-stopping when the envelope expires kills the long-lived-trust gap, because the danger isn't the suggestion, it's the standing permission that outlives the human who should've checked it.

Collapse
 
build996 profile image
build996 •

Most defenses in this thread check the name. The attack also has a tell that doesn't depend on the name at all: when the package came into existence. A squatted hallucination is younger than the model habit that invents it, so a CI gate that holds any new dependency whose first release is under 90 days old covers the long tail raised above without anyone harvesting names first. npm exposes time.created and PyPI's JSON API lists upload times per release, so it's a few lines. It stops working once a squatted package has aged past the threshold, which is exactly the 233-downloads-a-week case, so the allow-list still has to carry that one.

Collapse
 
james_anderson_h profile image
James Anderson •

This is the sharpest defense in the thread, because it sidesteps the name entirely — you don't need to know which hallucination, you just exploit the one invariant the attacker can't fake: a squatted package is always younger than the model habit that invents it. Gating any new dependency whose first release is under ~90 days old catches the whole long tail without harvesting a single name first, and it's a few lines against npm's time.created and PyPI's upload timestamps. The honest limit you flag is the right one: it fails exactly on the aged squat (the 233-a-week case), so age-gating and allow-listing aren't competitors — age handles the fresh long tail cheaply, the allow-list carries the patient attacker who waited out the window. Layer the two and you've covered both the lazy and the disciplined adversary. Stealing this.

Collapse
 
aidiveyt profile image
AI Dive •

The gate at the end is cheap to build. My 42 line hook on the tool call path, written to redact secrets, caught both Read and Bash output at a 27.9 ms round trip. The same seam takes an install command: check registry age and downloads, deny anything published last week.

Collapse
 
james_anderson_h profile image
James Anderson •

That's the whole argument made concrete — the expensive-sounding defense is a 42-line hook on the tool-call path, and the fact that it already caught both Read and Bash output at ~28ms proves the seam is general, not purpose-built. The install-gate is just another policy on the same chokepoint: intercept the command, check registry age and download count, deny anything a week old. The age-based rejection is especially nice there because it catches the fresh squat without needing to know the name in advance — the attacker can't make a just-registered package look old. The lesson underneath: once you have one enforcement point the model can't talk its way past, every new threat becomes a rule you add there rather than a system you rebuild. Cheap seam, reusable boundary. That's the part people underestimate because it isn't glamorous.

Collapse
 
indiainfranotes profile image
IndiaInfraNotes •

The package manager is a trust boundary, not a typing detail. A practical guardrail is to resolve the exact package and version through the registry API, fail closed on a miss, then pin the lockfile and review install scripts. That catches both hallucinated names and last-minute takeover attempts.

Collapse
 
james_anderson_h profile image
James Anderson •

Exactly — "trust boundary, not a typing detail" is the reframe that should reorganize how people think about this. The install line isn't a convenience to breeze past; it's the exact seam where untrusted names cross into your running system.

Your guardrail is strong because it fails closed: resolve the exact name and version through the registry first, and a hallucinated package dies at the boundary before anything executes — prevention, not detection. Pin the lockfile to kill the last-minute takeover; review scripts to bound what survives.

Best part: it's all deterministic. No classifier, no arms race. Exists and resolves — yes or no.

Collapse
 
docify profile image
Docify •

Hey, I'm building a small open-source CLI that analyzes a codebase and generates architecture/structure documentation. I'm looking for a few developers willing to run it against a real project and tell me where it gets things wrong.
You don't need to upload your code anywhere just run:
npx @autodocify/autodocs analyze .
Requires Node 20+.
If you try it, I'd especially like to know what it missed or misunderstood.

Collapse
 
kartik-nvjk profile image
Kartik N V J K •

The 19.7% hallucination rate across 16 models is bad enough, but the 43% that recur on every re-run is the detail that makes this weaponizable, an attacker only needs the name to be predictable. Registering 205,474 plausible package names isn't hard once the model keeps suggesting the same fakes. Are you screening install-time against a known-good allowlist, or catching these at PR review?

Collapse
 
james_anderson_h profile image
James Anderson •

You've put your finger on exactly why the 43% matters more than the 19.7% — randomness would protect you; predictability is what makes it a registerable target. On your question: allowlist at install-time is the stronger control, because PR review relies on a human recognizing an unfamiliar name, and the whole problem is that these sound legitimate. Catch it at the gate, not the eyeball.

Collapse
 
evanbright profile image
Evan Bright •

The part about treating AI-generated package names as claims rather than facts really stands out. I think dependency verification should become a normal part of any AI-assisted coding workflow, especially when agents can install packages automatically.

Collapse
 
james_anderson_h profile image
James Anderson •

Exactly — "verify before install" has to become reflex, not an afterthought, the moment agents can install on their own.

Some comments may only be visible to logged-in visitors. Sign in to view all comments.