What Is Bot Traffic? How to Detect and Stop It (2026)

Bot traffic is any visit to your website made by software instead of a person. Some of it you want, like search engines indexing your pages. Much of it you don't: scrapers copying your content, scripts probing your login page, bots filling in your forms, and fake clicks on your ads. Bad bot traffic inflates your analytics and puts junk leads in your CRM. When bots click your ads and then fill in your forms, it also teaches your ad platforms to go and find more bots.

GoodInbound Lead Screen catches the bots that get past your firewall: the ones that fill in your forms and click your ads.

• Every form submission labeled lead, review, spam, junk, or bot, with a one-line reason, from how the form was filled in, the network it came from, and the email and phone given.
• Works on the form you already have with one script tag (HubSpot, Webflow, WordPress, Framer, Marketo, or plain HTML), or through hosted forms, a headless form backend, a REST API, and an MCP server.
• Ad visits tracked by campaign, with IPs that arrive from your ads 5+ times in 2 hours and click IDs that land 3+ times in a day flagged for review.
• Only accepted leads sent back as conversions to Meta, TikTok, and Google Ads, so ad bidding learns from real people instead of bots.

The free plan screens 500 submissions a month with the behavior, network, and identity checks; ad visit tracking and the message check are part of Pro ($29/month). Start free, or read on for the fixes you can make yourself.

This guide covers what bot traffic is, how to tell good bots from bad ones, where bot traffic comes from, the signs to look for, and how to detect and stop it at each layer of your site.

What is bot traffic?

Bot traffic is every request to your website that comes from an automated program rather than a person using a browser. A bot can be a few lines of code that download a page, a crawler that follows every link it finds, or a headless browser: a real browser engine, run by a script with no screen, that loads pages, runs JavaScript, scrolls, clicks, and types.

The difference between those matters for what you can see:

• Simple bots send HTTP requests and never run JavaScript. They show up in your server and CDN logs, but not in Google Analytics or any other tag-based tool, because the tracking code never runs.
• Browser-based bots load your page the way a visitor would. They appear in analytics, fire your ad pixels, count as landing page visits, and can fill in and submit forms.

Bots don't need an invitation. Any public URL can be requested directly. Crawlers find new pages through links and sitemaps. And every ad you run puts your landing page in front of any bot that clicks it.

How much web traffic is bots?

Roughly half. Imperva's 2025 Bad Bot Report found that automated traffic made up 51% of all web traffic, passing human traffic, with bad bots at 37%. The announcement of its 2026 report puts bots at over 53% of web traffic, with 40% classed as malicious.

Imperva sells bot protection, and your share depends on what you publish and how you promote it. Treat these as an industry-wide picture rather than a benchmark for your own site.

Good bots vs. bad bots

Whether a bot is good depends on what it does and whether it plays by your rules. Good bots name themselves honestly in their user agent, follow robots.txt, and don't hammer your server. Bad bots pretend to be Chrome on Windows, ignore robots.txt, and go after your forms, logins, and ads.

What counts as good is up to you. A training crawler that follows robots.txt isn't malicious, but you may still not want it. An SEO tool's crawler is welcome when you run an audit, and less so when a competitor does.

The more useful split is between bots that respect your rules and bots that don't. The first group can be handled with robots.txt and allowlists. The second needs enforcement: firewall rules, rate limits, and screening at the points where bots do damage.

Where bot traffic comes from

Most sites get their bot traffic from the same handful of sources.

Crawlers and scrapers. Every public page gets found sooner or later. Search engines and AI crawlers arrive through links and sitemaps, and scrapers use the same routes to collect prices, content, and email addresses.

Vulnerability scanners. Automated scanners request common admin and config paths on every site they can reach, whether or not you run WordPress. They show up as bursts of 404 errors in your logs.

Form bots. Bots look for forms where they can post links, SEO offers, and fake sign-ups. Some post straight to the URL your form submits to, without loading the page. Others use a headless browser and fill in each field like a person would. How to stop contact form spam covers the different kinds.

Paid ads. An ad is an open invitation with a budget behind it. Bots click ads for several reasons: a publisher on a display or app network earns money from clicks and impressions, a competitor wants to drain your budget, or a crawler simply follows ad links. Placements outside the main platform are the usual weak spot, such as display networks and Meta's Audience Network, which shows your ads inside other companies' apps (see Meta Audience Network). Because the bot arrives through your ad, its visit carries a click ID like gclid or fbclid and looks exactly like paid traffic. Ad fraud covers this side in depth.

Your own tools. Uptime monitors, SEO crawlers, page speed tests, and the link previews Slack or LinkedIn build when someone shares your URL are all bots. They're harmless, but they count.

Bought traffic. Services that sell website visits by the thousand deliver automated visits. They raise the visitor count and never turn into customers. If a service like that is ever pointed at your site, its visits look just like the bot traffic described here.

Signs of bot traffic on your site

No single metric proves a visit came from a bot. Look for several of these together, especially when they start on the same day.

In your analytics

• Spikes with no cause. Traffic jumps without a campaign, launch, press mention, or email behind it.
• Sessions with no engagement. GA4 counts a session as engaged when it lasts longer than 10 seconds, has a key event, or has 2 or more page views (Google's definition). Bot sessions usually do none of these, so a source whose engagement rate suddenly drops, with near-zero engagement time, is a suspect.
• Locations you don't serve. A surge from a country or city where you have no customers, often to a single page.
• Uniform technology. One browser version, operating system, or screen resolution suddenly making up most of a spike.
• Direct or unassigned traffic to one page. Bots often arrive without a referrer, so they land in the Direct or Unassigned channel.

GA4 already removes traffic from known bots and spiders, using Google's own research and the IAB's International Spiders and Bots List. You can't turn this off, and you can't see how much it removed (Google Analytics Help). What's left is the bot traffic that doesn't announce itself. How to find and filter bot traffic in Google Analytics walks through the GA4 reports step by step.

In your paid traffic

• Clicks rise, engaged visits don't. Ad clicks and spend go up while engaged sessions and leads stay flat.
• The same visitor over and over. One IP address arriving from your ads many times in a short window, or one click ID landing several times in a day, which a person rarely does.
• Placements you didn't choose. A large share of clicks from app placements, display partners, or search partners, with no leads to show for it.
• Conversions that never become customers. Form fills from ad traffic that your sales team can't reach.

In your forms

• Submissions seconds after the page loaded, or at a steady rhythm through the night.
• Gibberish or mismatched details: random-letter names, disposable email domains, phone numbers like 555-0100, or a phone number from a different country than the visitor.
• Repeats from one IP address, or many submissions with near-identical text.
• Links in fields that shouldn't have them, such as the name or company field.

To check a suspicious one, paste the submission into the free form spam checker, the address into the fake email checker, or the number into the phone number validator.

In your server logs

• Request rates no person could produce, such as hundreds of requests a minute from one IP.
• Scripting tools in the user agent, like python-requests, curl, or Go-http-client, or no user agent at all.
• Bursts of 404s for paths you don't have, like /wp-login.php on a site that isn't WordPress.
• Visitors calling themselves Googlebot from IP addresses that don't belong to Google.

How to detect bot traffic

The signs tell you something is off. These four checks tell you what the bots are and where they come from, without an enterprise bot management contract.

1. Read your server or CDN logs

Analytics only sees visitors that run your tracking code. Your logs see every request. On a server using the default nginx or Apache combined log format, these one-liners show the heaviest hitters:

An IP with thousands of requests, a user agent that names a scripting library, or a cluster of 404s on admin paths is a bot. If your site sits behind a CDN, use the CDN's analytics instead: your server logs show the CDN's IP addresses unless you've set it up to record the visitor's.

2. Check that Googlebot is really Google

Scrapers often call themselves Googlebot, because few sites block it. Google's verification method is a two-step DNS check: run a reverse lookup on the IP, check that the hostname ends in googlebot.com, google.com, or googleusercontent.com, then run a forward lookup on that hostname and confirm it returns the original IP.

Google also publishes its crawlers' IP ranges as JSON files, which is easier to check at scale. A "Googlebot" that fails this check is a scraper using a borrowed name, and safe to block.

3. Line up analytics, logs, and leads

Put four numbers side by side for the same period: requests in your logs, sessions in analytics, form submissions, and leads that turned out to be real. Where the spike appears tells you which kind of bot you have:

• In your logs but not in analytics: simple bots that don't run JavaScript. They cost you bandwidth, not data.
• In analytics but not in submissions: browser bots browsing. They distort your reports and your landing page tests.
• In submissions but not in real leads: bots reaching your forms. This is the expensive kind, because those submissions land in your CRM and often in your ad conversions too.

4. Look at paid traffic on its own

Break ad traffic out by campaign and placement, then compare clicks, engaged visits, and real leads for each. In Google Ads, add the Invalid clicks column: open Campaigns, click the Columns icon above the table, search for "Invalid clicks", and click Apply. Google doesn't charge for invalid traffic it detects before billing, and credits what it finds afterwards (Google Ads Help). The clicks Google didn't flag are the ones to check yourself.

GoodInbound's Paid traffic view does this split for you. It records visits that land from an ad with a click ID (Google gclid, gbraid, or wbraid, Microsoft msclkid, Meta fbclid, LinkedIn li_fat_id, or TikTok ttclid) or paid UTMs. For each campaign it shows ad visits, the share that engaged, leads, and accepted leads. IPs with 5 or more ad visits in 2 hours and click IDs that land 3 or more times in a day are flagged. Ad visits are part of Pro, which includes 1,000 analyzed ad visits a month. More in click fraud protection.

How to stop bot traffic on your website

There's no single switch. Each layer stops a different kind of bot, and the bots that get furthest are the ones that cost you the most. Work from the outside in.

1. Tell well-behaved crawlers what's allowed with robots.txt

robots.txt is a text file at the root of your site that tells crawlers which paths they may fetch. Reputable crawlers follow it; others don't. Google describes it as mainly a way to keep crawlers from overloading your site, and says it isn't a way to keep a page out of Google, which needs noindex instead (Google Search Central).

The robots.txt decision most sites face now is AI crawlers. The major AI companies run separate bots for training and for search, so you can opt out of model training and stay visible in AI answers:

The Allow groups are optional, since anything you don't disallow is allowed, but they make your intent clear. What each bot does:

• OpenAI runs GPTBot to collect training data and OAI-SearchBot for ChatGPT search. Sites that block OAI-SearchBot aren't shown in ChatGPT search answers (OpenAI).
• Anthropic runs ClaudeBot for training, Claude-SearchBot for search, and Claude-User for pages a user asks about, and says all three honor robots.txt (Anthropic).
• Common Crawl's CCBot collects pages for Common Crawl's open archive of the web, and Common Crawl tells site owners to block it by name in robots.txt (Common Crawl).
• Google-Extended isn't a separate crawler. It's a robots.txt token that controls whether Google can use your content to train Gemini models and for grounding, and it has no effect on your inclusion or ranking in Google Search (Google).
• User-triggered fetchers such as ChatGPT-User and Perplexity-User may not follow robots.txt, because a person asked for the page (OpenAI, Perplexity).

robots.txt only works for bots that choose to read it. Everything below is for the ones that don't.

2. Block scanners and scrapers with WAF rules

A web application firewall (WAF) checks each request at the edge, before it reaches your server. Most CDNs and many hosts include one. Cloudflare is a common example, and its free plan includes:

• Bot Fight Mode, which challenges requests that match bot patterns. Turn it on under Security → Settings, filtered by Bot traffic. It can also challenge API and mobile app traffic, and custom rules can't skip it, so test your forms, webhooks, and integrations afterwards (Cloudflare docs).
• AI bot traffic policies, which block or allow search, agent, and training bots separately, either on all pages or only on pages that show ads (Cloudflare changelog).
• Five custom rules (Cloudflare docs), enough for the basics below.

Three custom rules worth adding. First, block probes for files you don't have, with action Block (leave out the WordPress paths if you run WordPress):

Second, block scripting tools that don't bother to disguise themselves, with action Block. Exclude any paths that receive webhooks or API calls first, since some legitimate services use these libraries:

Third, challenge page views from hosting networks, where very few real buyers browse from, with action Managed Challenge:

Those numbers are the networks of DigitalOcean, OVH, and Hetzner; add the ones that show up in your own logs. The GET condition keeps the rule off form posts, because a background request can't complete a challenge; exclude your API paths for the same reason. The not cf.client.bot part lets Cloudflare's list of known good bots through. Cloudflare warns that block and challenge rules based on path, ASN, or country can otherwise catch those bots and hurt your SEO and monitoring (Cloudflare WAF FAQ).

Other WAFs offer the same kinds of rules under different names, so the logic carries over. One thing to avoid is long lists of blocked IP addresses. Bots rotate through IPs, and residential proxy networks route them through ordinary home connections, so an IP blocklist grows forever and still misses the next one.

3. Rate-limit forms, logins, and search

Rate limiting caps how many requests one IP can make in a window. It stops floods and password-guessing without affecting normal visitors. Put limits on the endpoints bots abuse: form submissions, login, password reset, search, and anything that costs you money each time it runs.

On Cloudflare's free plan you get one rate limiting rule, counted per IP over 10 seconds; paid plans add more rules and longer windows (Cloudflare docs). On your own nginx server, the limit_req module does the same job. This example limits form posts to 5 a minute per IP and leaves page views alone, because requests with an empty key aren't counted:

nginx rejects requests over the limit with a 503 by default; limit_req_status 429 returns the more accurate "Too Many Requests" (nginx docs).

Rate limits stop volume, not quality. A bot that sends one fake lead per IP through a proxy network never trips them.

4. Use bot management when bots are a security problem

If you run a store, customer logins, or anything with limited inventory, bots become a security problem: credential stuffing, card testing, and scalping. That's what bot management products are for. They score each request on browser fingerprints, behavior, and network reputation, then block, challenge, or allow it by score. CDN providers sell them as add-ons and specialist vendors sell them on their own. They're built for large sites: Cloudflare's Bot Management, for example, requires an Enterprise plan (Cloudflare docs).

For a B2B site whose bot problem is fake leads and wasted ad spend, you rarely need one. What hurts you is the small share of bots that gets through to your forms and your ad conversions, which is what the last two layers deal with.

5. Screen form submissions before they reach your CRM

Firewalls and rate limits judge requests. They can't tell whether a form submission that looks human came from a real buyer, and the bots that reach your forms are the ones built to look human: a headless browser on a home IP address, typing into each field. Honeypots and time checks stop the simple ones (how to stop contact form spam covers them). CAPTCHAs stop more, though the visible ones add friction for every real visitor.

Screening checks every submission on several signals at once. GoodInbound Lead Screen labels each one lead, review, spam, junk, or bot, with a one-line reason, based on:

• Behavior: how long the form took to fill in, whether anyone typed, signs of automation, and pasted text.
• Network: datacenter and hosting networks, VPNs, repeat submissions from one IP, and countries you've blocked.
• Identity: email addresses that are invalid, disposable, free, role-based, gibberish, or on a domain with no mail server (email validation API), and phone numbers that are invalid, follow fake patterns like 555 numbers, or are VoIP, premium rate, or from a different country than the visitor (phone validation API).
• Message (Pro): reads what was written and catches sales pitches and junk, whether a person or AI wrote it (how to stop sales pitches through your contact form).

Field checks answer in under 500 ms. Lead Screen works on HubSpot, Webflow, WordPress, Framer, Marketo, and plain HTML forms with one script tag, and through GoodInbound hosted forms, a headless form backend, the REST API, and the MCP server. Because every submission arrives with a label, bots and junk can stay out of your CRM and your sales team's queue. The free plan covers 500 submissions a month with the behavior, network, and identity checks; the message check is part of Pro.

6. Keep bots out of your ad conversions

This is where bot traffic costs the most. Google Ads smart bidding and Meta's ad delivery optimize toward whatever you report as a conversion. If bots click your ad, fill in your form, and every submission fires a Lead event, the platform learns that those clicks convert and buys more of them.

Three fixes:

1. Send conversions for real leads only. Fire the conversion after the submission has been screened, not on the thank-you page. Lead Screen sends only accepted leads back: natively to the Meta Conversions API (a Lead event with hashed email and phone, fbc and fbp, deduplicated with your pixel by event ID) and the TikTok Events API, and to Google Ads through a goodinbound_lead event in Google Tag Manager or a webhook.
2. Cut the placements that send clicks but no leads. Compare ad visits, engaged visits, and accepted leads by campaign, and pause or exclude the placements and partners that never produce a real lead.
3. Exclude repeat offenders by hand. Google Ads lets you exclude up to 500 IP addresses per campaign under the campaign's Settings → Additional settings → IP exclusions. Campaign-level IP exclusions aren't available for Performance Max, video, App, hotel, or Smart Display campaigns; account-level exclusions cover Performance Max (Google Ads Help). In Lead Screen, Exclude on a flagged IP copies it with the steps to add it in Google Ads. Nothing is excluded automatically, and because bots rotate IPs, treat exclusions as cleanup rather than protection.

For more on the paid side, see click fraud protection and how Lead Screen compares with ClickCease, CHEQ, and Anura.

Which fix stops which bots

FAQ

What does bot traffic mean?

Bot traffic means visits and requests to a website that come from software instead of people. It includes useful bots, like search engine crawlers and uptime monitors, and harmful ones, like scrapers, spam bots, and fake ad clicks.

Is it true that 51% of internet traffic is bots?

By Imperva's measure, yes. Its 2025 Bad Bot Report found automated traffic made up 51% of all web traffic, with bad bots at 37%, and its 2026 report puts bots at over 53%. These are industry-wide figures from a bot protection vendor. Your own site's share can be much higher or lower.

How do I identify bot traffic?

Look for traffic spikes with no cause, sessions with no engagement, visits from places you don't serve, many requests from one IP, scripting tools in user agents, and form submissions sent seconds after the page loaded. Confirm in your server or CDN logs, since simple bots never show up in analytics.

How do I stop bot traffic on my website?

Use layers. robots.txt handles well-behaved crawlers, WAF rules handle scanners and scrapers, and rate limits protect forms and logins. Screening every form submission catches the bots built to look human, so they don't reach your CRM or your ad conversions.

How does bot traffic affect ad performance?

Bots that click your ads spend your budget, and Google only filters out or credits the invalid clicks it detects. Bots that also fill in your form create fake conversions, and bidding systems that optimize for conversions then look for more visitors like them. Your cost per lead can look healthy while the number of real leads falls. Screening submissions and sending only real leads back as conversions breaks that loop.

Does Google Analytics filter out bot traffic?

Partly. GA4 automatically excludes traffic from known bots and spiders, and you can't turn that off or see how much was removed. Bots that pose as normal browsers still get counted. How to find and filter bot traffic in Google Analytics shows how to find what's left.

Is it illegal to bot websites?

Running a bot isn't illegal in itself: search engines do it all day. What the bot does can be. Logging into accounts with stolen passwords, testing stolen cards, or faking ad clicks to drain a budget or earn a payout can break computer misuse and fraud laws in many countries, and scraping can breach a site's terms of service. The rules depend on the country and the case, so ask a lawyer about a specific situation.