How to Filter Bot Traffic From Analytics (and Why GA4 Misses Most of It)

The short answer

There are three kinds of non-human traffic, and you filter each one differently. The polite crawlers announce themselves in the user-agent string, so any tool can drop them. The automation frameworks run real JavaScript and look almost human, so you match them on known signatures. The stealth ones send a clean browser fingerprint from a rented server, and the only reliable tell is the network they arrive from. Do all three at the point where data enters your system, not in a report afterward, because a hit you never stored cannot skew a number later.

Google Analytics does the first job and skips the other two. That is the whole reason your dashboard shows visitors from cities you do not sell to, sessions that last zero seconds, and spikes that never buy anything. If you remember one thing from this page: the useful automated hits to catch are the ones that lie about being machines, and no public list will ever contain them.

The three kinds of bots you are actually fighting

I find the whole thing easier to reason about once you split it by how hard each type is to see.

• Self-identifying crawlers. Googlebot, Bingbot, Ahrefs, Screaming Frog and thousands more put a token like bot, crawler or spider in their user-agent. A library such as isbot removes them in one line. This is the easy tier and every serious tool handles it.
• Headless browsers and automation tools. Puppeteer, Playwright, Selenium and headless Chrome execute JavaScript, so they fire your tracking tag exactly like a person. They usually still leak: the word Headless in the user-agent, a navigator.webdriver flag set to true, or a raw client string like python-requests or go-http-client. You catch these with pattern matching, not a public list.
• Stealth traffic from datacenter IPs. This is the hard tier. A scraper on a rented cloud box can send a pristine Chrome user-agent and pass every check above. What it cannot easily fake is its network. Real people browse from residential and mobile ISPs. Almost nobody loads your site from an Amazon, Google Cloud, Azure, Hetzner or OVH address, so the network behind the IP becomes the signal.

Why GA4 misses most of them

GA4 removes known bots automatically, using a mix of Google's own research and the IAB/ABC International Spiders and Bots List. That covers the first tier well. The problem is what Google's own documentation admits about control.

At this time, you cannot disable known bot traffic exclusion or see how much known bot traffic was excluded.

So you get exactly one filter, you cannot tune it, and you cannot even see how much it took out. More to the point, a known-bots list can only contain machines that identify themselves. Headless Chrome presenting a normal user-agent, and a scraper on a datacenter IP, are on no list at all, because from the outside they look like a visitor. They run your tags, count as sessions, and quietly inflate everything downstream from bounce rate to conversion rate.

This is not a theory I am asking you to take on faith. Plausible ran a controlled test, pointing Puppeteer-driven requests at a dummy site, and Google Analytics recorded the fake pageviews in all three scenarios they tried, including one sent from a datacenter IP. Those are their numbers on their own test, and it favours their product, so read it as a directional result rather than gospel. It matches what I see in the field.

How to spot the pattern in your own dashboard

You do not need a specialist tool to find this. Open your analytics and look for the fingerprints automation leaves behind.

• Zero-second sessions clustered on one fingerprint. A machine loads a page and leaves in the same instant, over and over, from what looks like a single visitor.
• One browser and operating system combination, often an outdated version, driving an unrealistic share of your hits.
• Direct visits with no referrer landing deep inside your site, on URLs no human would type from memory.
• Sudden volume from a country or city that has never converted and matches no campaign you ran.
• IP addresses that resolve to a hosting company. Paste a suspicious one into any ASN lookup, and if it comes back as AWS, Google Cloud or a data center, it is almost certainly a machine.

None of these is proof on its own. Together they are a pattern, and once you have seen it a few times you stop trusting a raw pageview count that has not been filtered first.

How Conclick filters bots in three layers

I build Conclick, which is built on the open-source Umami engine, so treat this as the founder telling you how his own tool works rather than a neutral survey. Umami ships the first line of defence. The automation and datacenter layers are what I added on top, because the known-bot filter alone was not enough for the customers I care about.

Every incoming hit passes three server-side checks before anything is written to the database, and a rejected hit gets a tiny response and is never stored:

• Layer one drops self-identifying crawlers with the isbot library, and also rejects any request that arrives with no user-agent at all, which is a classic sign of a raw script.
• Layer two matches a list of automation and HTTP-client signatures: headless, puppeteer, playwright, selenium, python-requests, go-http-client, scrapy, wget and their relatives. The browser-side tracker also refuses to send when navigator.webdriver is set, so a lot of automation dies before it ever reaches the server.
• Layer three looks up the IP's autonomous system in a MaxMind ASN database and drops the hit if the network belongs to a cloud or hosting provider. This is the layer that catches the stealth traffic with a spotless user-agent, and it is the one most privacy-first tools skip.

What you can do today, whatever tool you use

You do not have to switch tools to improve this. A few practical moves get you most of the way:

• In GA4, add IP exclusions for your own office and known offenders, build a segment that removes zero-engagement sessions, and validate the hostname so hits from cloned or spoofed copies of your domain drop out of the main view.
• Judge suspicious visits by their network, not just their user-agent. An ASN lookup takes ten seconds and settles most arguments about whether something is a person.
• Run two analytics tools on the same site for a week and compare the totals. The gap between them is roughly the automated traffic one of them is still counting.
• Do not chase one hundred percent. Sophisticated invalid traffic rotates through residential proxies and mimics human timing, so any honest figure you land on is a floor, not the whole picture. The goal is decisions you can trust, not a spotless funnel.