Three bot-filtering heuristics that look correct and delete real users

I spent a week writing a filter for junk queries hitting a search endpoint. Writing the filter was the easy part. The hard part was that three heuristics I was completely confident about each deleted real users, and I only found that out because I ran them against production traffic before shipping.

Here is what the junk actually looks like, why the obvious rules fail, and the one property that turned out to separate the two.

The traffic came from WooCommerce shops. Last week one query in a shop's log was this:

uFbX'kCSAQp<'">)yLNODq

Nobody types that.

It is a canary, not a typo

That string has two halves. uFbX is a random four-character marker. The rest is a payload that tries to break out of a quote, then out of an HTML attribute, then out of a tag, all in one go.

The marker is the interesting part. The scanner injects a token it made up so it can then read the page that comes back and look for its own string. If the token appears somewhere it should not, the tool has found a place where visitor input reaches the page unescaped. That is the whole test.

It never arrives alone. In one window of that shop's log, twenty five of thirty consecutive queries were this traffic: eight markers, three shots each, plus a single bare probe.

the bare token uFbX
token + punctuation uFbX.\"(,.(.(\')
token + polyglot uFbX'kCSAQp<'">)yLNODq

Eight tokens, a few minutes. All eight payloads shared an identical five-character core, which is what makes them recognisable at all.

Why your shop

Because /?s= is public, unauthenticated and present on every WordPress installation ever made. It takes a string, does something with it, and renders a page. From a scanner's point of view that is a text input on an endpoint with no login in front of it, which is precisely what it is hunting for.

Nothing about this is targeted. Nobody picked your shop. It is a crawler working down a list, and I can see the same shapes arriving at unrelated accounts within the same hours. Bare years as the entire query, for instance, turned up on seven unconnected stores.

One store I looked at had no visible search box on the page at all. It was still collecting queries.

You almost certainly cannot see it

Here is the part that makes this invisible rather than merely annoying: WordPress does not keep a record of what people search for.

Core runs the query and renders the results page. Nothing is written down. There is no admin screen listing yesterday's searches, because there is no table behind it. Every plugin that shows you a search report is a plugin that started storing them, from the moment you installed it and not one day earlier.

Analytics does not reliably fill the gap either. Site search tracking generally depends on JavaScript executing in a browser, and a good deal of this traffic never executes any. Whatever does get through then lands in the same bucket as your customers, unlabelled.

Your server access log has all of it, in raw form, with every ?s= parameter intact. Nobody reads server access logs, and they rotate away.

So the honest answer to "has this been happening to me" is: yes, and you have no record of it. The shops that can see it are the ones running something that writes queries down.

What it actually costs you

Not security. These strings are harmless against a site that builds its database queries properly, which is the default in WordPress and WooCommerce. The string goes in as a parameter and comes back out as text. Anyone who shows you one of these and tells you that you have been breached is selling something.

The real costs are quieter, and there are three.

Your search analytics stop meaning anything. "What are people searching for and not finding" is one of the most direct signals a shop has about its own catalogue. It tells you what to stock, what to rename, what your customers call things. If a quarter of your log is a crawler, that signal is buried under noise that never bought anything. On one account roughly half the traffic was junk in alphabets the shop does not sell in. Any conclusion drawn from that log without stripping the noise first is a confident conclusion about nothing.

On metered search, you pay for it. Any service that charges per query will happily count a scanner's two dozen shots as two dozen searches, unless it filters them out before counting. Worth asking whoever you are paying.

And it hides the occasional real problem. On one shop, the search returned a page from the store's own catalogue that had no business being there: a doorway page, injected into the site, sitting indexed alongside the products. The noise was not the attack. But a log nobody can read is also a log where something like that goes unnoticed for months.

How to check your own log

If you do have query logging, these are worth a look:

strings containing < > or runs of quote marks
long unbroken runs of punctuation
the same 4-character prefix repeated two or three times in a minute
queries in an alphabet your customers do not use
a bare year as the entire query: "2026", "2027"

None of these needs a tool. Sort by query, eyeball the top of the list, and the pattern is usually obvious within a minute.

What not to treat as a signal

This is the half that most guides get wrong, and it is the half that matters, because a filter that eats real customers is worse than no filter.

Every one of the three rules below looks sensible. All three are wrong, and I know they are wrong because each of them fired on genuine traffic in my own logs.

An apostrophe on its own means nothing.

l'oreal men's o'neill

Apostrophes are in brand names, and they are in plain English. Treating a quote character as hostile deletes a slice of your real searches. It is the sequence and the surrounding punctuation that carries the signal, never the single character.

A string with no vowels is not obfuscation.

FT3015 D50B0 epson emc800

Those are catalogue numbers, and customers type them constantly. Model codes, SKUs and part numbers are the most valuable queries in a lot of shops, because someone typing a part number already knows what they want. A "looks like line noise" heuristic throws away your highest-intent traffic first.

A shared prefix is usually a person.

fantic → fantic competition

Two queries, seconds apart, one a prefix of the other. That reads like a script iterating. It is a human being who searched a brand, saw too much, and added a word. Refining a search is what good searching looks like. The scanner pattern is a fixed random token repeated with different junk appended, which is the opposite shape: the constant part is meaningless and the variable part is the payload.

The general rule underneath all three: look for structure that no human would produce, not for characters that look unusual. Unusual characters are what your catalogue is made of.

What we did about it

We went through the payloads we had actually received, pulled out the shapes they share, and now match against those. A query that matches is answered with an empty result and stops there. It is not counted against the shop's plan, and it does not get written into the search history, so it cannot pollute the analytics later.

That is deliberately a short list built from observed traffic rather than a clever general rule. The negative cases above are why: every heuristic that sounded good in the abstract turned out to eat real customers, so the bar for adding a pattern is that we have seen it arrive and that it cannot occur in a product name.

You cannot stop this at the source. /?s= is public by design and a crawler does not need permission to use it. What you can do is make sure it never reaches the two places where it does damage: your analytics, and your bill.

If you are running your own store, the useful takeaway is not the filter. It is that the log you have been reading, if you have one at all, probably has a quarter of its rows contributed by something that was never going to buy anything, and that your conclusions about what customers want should be drawn from the rest.