---
title: "Only 1 in 4 Claimed AI Bot Hits Verified: What 30 Days of Edge Logs Mean for Agency Reports"
description: "In 30 days of groas edge logs, 25% of claimed AI bot hits on robots.txt verified. Here’s what agencies can defend in a client report—and what they can’t."
url: "https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/beac49ac-e479-4c0c-90de-58244721019e.png"
published: "2026-10-10T05:28:49.714Z"
modified: "2026-10-10T05:28:49.773Z"
---

October 10, 2026 · 9 min read

# Only 1 in 4 Claimed AI Bot Hits Verified: What 30 Days of Edge Logs Mean for Agency Reports

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A bouncer's UV torch checks four look-alikes all wearing 'GPTBot' name tags; only one hand shows a glowing stamp, so three of four claims are fake.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/beac49ac-e479-4c0c-90de-58244721019e.png)

In this article

1. [Start with the claim the edge log can test](#start-with-the-claim-the-edge-log-can-test)
2. [The homepage has three numbers, not one](#the-homepage-has-three-numbers-not-one)
3. [Deep-page counts shrink further after verification](#deep-page-counts-shrink-further-after-verification)
4. [Three paths account for nearly all retrieval fetches](#three-paths-account-for-nearly-all-retrieval-fetches)
5. [Why the standard dashboard overstates what it knows](#why-the-standard-dashboard-overstates-what-it-knows)
6. [Put these figures in the client report](#put-these-figures-in-the-client-report)
7. [Before you roll a platform across ten domains](#before-you-roll-a-platform-across-ten-domains)

On groas.com, **only 3,419 of 13,835 claimed AI bot hits on `robots.txt` verified** over 30 days. That is roughly one in four. If an agency reports the other three as visits from named AI vendors, it is asking the client to trust a User-Agent string that any script can write.

The problem follows the report down the site. About 23% of claimed homepage hits verified; deep blog pages ranged from 2% to 8%. And roughly 98% of verified retrieval-agent fetches went to just three paths. Those figures do not tell me which pages appeared in answers. They do tell me that a raw bot-hits chart is a poor account of what happened.

## Start with the claim the edge log can test

This is **our own 30 days of Cloudflare edge data for groas.com**, not a benchmark for every site. I logged requests claiming an AI bot User-Agent and checked the source IP against the named vendor’s published ranges. A match counted as verified. I did not use reverse DNS as a fallback; when I had no published range to check, I marked the request unverifiable, not verified.

That distinction matters. _Unverified_ does not mean _proven spoofed_. It means the log does not support the vendor identity printed in that row. For an agency putting a vendor name in a client report, that is already a serious limit.

| Path          | Claimed AI bot hits | IP-verified hits | Verified share |                          Verified retrieval-agent fetches |
| ------------- | ------------------: | ---------------: | -------------: | --------------------------------------------------------: |
| `/robots.txt` |              13,835 |            3,419 |          24.7% | Discovery path; not reported here as an answer-page count |
| `/` homepage  |               9,202 |            2,128 |          23.1% |                                                       962 |

_Source: groas.com Cloudflare edge logs, 30-day window; User-Agent claims checked against published vendor IP ranges. Percentages are verified hits divided by claimed hits on the same path. The retrieval-agent column is a narrower count, not a count of displayed answers._

![Bar chart comparing claimed AI bot hits with the smaller IP-verified share](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/17f78865-062a-4106-b314-0a4c20656c4f.png)

A User-Agent is useful for sorting requests. It is not identity verification. OpenAI, for example, publishes [separate verification files for three agents](https://www.honeyb.ai/blog/gptbot-crawler-guide), so even a familiar bot name requires the right check. I used to give the string more credit when scanning logs. I would not put it in a client-facing AI visibility total without checking the IP now.

Other studies illustrate why the distinction is worth making, but they are not direct comparisons with our percentages. A [two-week study of 16 AI crawlers](https://www.humansecurity.com/learn/blog/ai-crawler-spoofing-chatgpt-mistral-perplexity/) found 5.7% of traffic bearing an AI-crawler User-Agent was spoofed, rising to 16.7% for ChatGPT-User. On [one small unpromoted domain](https://digimirror.io/your-bot-traffic-logs-are-lying-to-you/), only 6 of 33 claimed live assistant fetches verified. Different sites and methods produce different numbers. My edge log supports the narrower conclusion: **most claimed hits in these groas path counts could not be verified as the vendor they named.**

## The homepage has three numbers, not one

The homepage received 9,202 claimed hits. IP checks brought that down to 2,128 verified hits. Filtering for retrieval agents brought it down again to 962 fetches. Each step answers a different question; none is a substitute for the next.

- **9,202 claimed hits:** requests whose User-Agent named an AI bot.
- **2,128 verified hits:** requests whose IP matched the relevant published vendor range.
- **962 retrieval-agent fetches:** verified homepage requests from agents used to retrieve content for answers, rather than training crawls.

[Retrieval agents such as OAI-SearchBot and ChatGPT-User serve a different role from training crawlers such as GPTBot](https://digimirror.io/your-bot-traffic-logs-are-lying-to-you/). If I combine the two in a single answer-visibility number, a crawl that cannot itself produce a citation makes the report look better. The split is more useful than the bigger total.

Still, I would not tell a client that 962 homepage fetches produced 962 answers, prompts, or citations. The edge log records requests to our site. It does not show the assistant’s final response or tell us whether the fetched page was displayed as a source. **A verified retrieval fetch is evidence of access, not proof of a citation.** That is the number I would report, with that limit beside it.

## Deep-page counts shrink further after verification

The homepage was not the worst case. In the same 30 days, our category-style pages each drew around 3,000 claimed hits, but their verified share sat near 7%. They recorded no retrieval-agent fetches in the count used here. One deep post, `/post/flat-rate-aeo-pricing-per-domain`, logged 340 claimed hits and 8 verified: 2.4%. Across deep blog pages, the verified share ranged from **2% to 8%**.

Those figures come from the same groas.com edge-log checks as the homepage table. The range matters more than a single blended average: it shows how little a raw page-level hit count tells me about a particular URL. A dashboard could give that post a bar for 340 ‘AI visits.’ I could substantiate 8 vendor-IP matches. I would not quietly rename the remaining 332 as visits from that vendor, nor claim I had established what each unverified request was.

There is a reason to be cautious about odd traffic on deep URLs. On a [small unpromoted test domain](https://digimirror.io/your-bot-traffic-logs-are-lying-to-you/), claimed ChatGPT-User requests went looking for `.env.production` and `secrets.yaml` rather than useful pages. The [study of 16 AI crawlers](https://www.humansecurity.com/learn/blog/ai-crawler-spoofing-chatgpt-mistral-perplexity/) describes spoofing patterns that make a familiar User-Agent and a quick glance at a log insufficient checks. Neither source proves that every unverified request in our logs was an attacker. It does explain why I refuse to treat a claimed name as a verified reader.

The practical decision is simple: **verify before ranking pages by AI bot traffic.** Otherwise the pages with the busiest unverified logs can look like the biggest visibility wins.

## Three paths account for nearly all retrieval fetches

After IP verification and the retrieval-agent filter, about **98% of fetches in our 30-day edge data went to three paths**: `robots.txt`, the homepage, and one changelog post. The homepage recorded 962 retrieval-agent fetches and the changelog post 232. The other 197 site paths shared roughly 2% of the verified retrieval-agent fetches.

_Source: the same groas.com 30-day edge logs, grouped by path after IP and agent-type checks. The 98% figure includes requests to `robots.txt` for discovery; it must not be read as the share of answer-page fetches, much less the share of citations._

That last qualification is not fine print. A fetch of `robots.txt` tells me an agent checked the site’s rules. It does not tell me the assistant read that file to answer a customer. Even the homepage and changelog counts stop at the request. I can say retrieval agents reached those pages; I cannot use edge logs alone to say how often the pages appeared in responses.

![Homepage and changelog drawing most delivery trucks while other site paths receive few](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/64af3ed7-0763-4386-9be7-9e1ed97b4ea9.png)

I used to tell clients that coverage meant getting every page cited. I was wrong to make that the starting promise. On this site, the measurable activity was highly concentrated, and a long list of URLs with claimed bot hits would have hidden that fact. **Name the paths that earned verified retrieval fetches first.** Make the case for work on the remaining pages with evidence, not with a fuller-looking chart.

## Why the standard dashboard overstates what it knows

The failure starts at collection. A script can set its User-Agent to `GPTBot` or `PerplexityBot`; the receiving server will log that text. To support the identity claim, a report needs a further check against the vendor’s published ranges, with any additional vendor-specific verification kept distinct. Skip that step and the chart measures claims, including claims the agency cannot substantiate.

It can also mix incompatible activity. A training crawl is not a retrieval fetch, a retrieval fetch is not a displayed citation, and a displayed citation is not attributable revenue. Each may matter for a different question. None should borrow the label of the next just because the graph looks sparse otherwise.

Some activity is also outside what this method can separate. As [this account of bot-log limits explains](https://digimirror.io/your-bot-traffic-logs-are-lying-to-you/), Googlebot crawling and the Google-Extended permission flag do not give an edge-log reader a separate Gemini fetcher to count. I would rather leave that gap visible than fill it with an invented ‘Gemini visits’ total. A [test of top domains](https://ppc.land/full-bot-protection-drops-to-2-4-of-popular-websites-datadome-finds/) found two in three blocked none of ten test bots, another reminder that site-level bot controls and bot-identity reporting are different jobs.

So I do not want a grand total called _AI visits_. **I want a table whose columns say what was checked and what remains unknown.** It is less theatrical. It is also a report a client can challenge without making the whole thing fall over.

## Put these figures in the client report

For a client, I would show three connected views: verified fetches by engine, verified retrieval-agent fetches, and the paths receiving those fetches. The first tests identity. The second separates agents that retrieve content for answers from training crawlers. The third shows whether activity is spread across the site or concentrated somewhere the team can inspect.

| Report line                       | What I would show                                                                    | What I would not claim                                      |
| --------------------------------- | ------------------------------------------------------------------------------------ | ----------------------------------------------------------- |
| **Verified fetches by engine**    | Claimed and IP-verified requests, with the verified share for each engine            | That a blended bot total establishes each engine’s activity |
| **Retrieval-agent fetches**       | Verified requests from retrieval agents, separate from training crawls               | That each fetch produced an answer or citation              |
| **Paths receiving those fetches** | URL-level counts, led on our site by the homepage’s 962 and the changelog post’s 232 | That every path with raw bot hits was used in an answer     |

_The groas counts come from the 30-day edge-log window above. An [agency tracker comparison](https://www.airanklab.com/blog/best-aeo-trackers-agencies-marketing-teams) also discusses engine breakdowns and cited-page analysis, but a cited-page measure requires evidence beyond an edge request._

For a client asking why a 200-path site has activity concentrated in a few places, this table is more helpful than a promise of blanket coverage. It shows where the requests went and where the evidence stops. To run the checks on a client domain, start with the [curl and CDN audit](https://groas.com/post/can-chatgpt-actually-read-your-site-a-45), then use the [edge-log tutorial](https://groas.com/post/tutorial-prove-which-client-pages-ai-bot) for the IP-verification method. Neither step turns a server request into a citation; that boundary belongs in the report too.

## Before you roll a platform across ten domains

You are choosing a reporting standard you will repeat across clients, not just a dashboard your team likes in a demo. Run this checklist on a trial domain first. The pricing question deserves the same scrutiny as the data: the [AEO tools comparison](https://aissist.io/insights/best-ai-visibility-tools) lists a $99-per-domain, 25-prompt base for Semrush and tier or add-on limits for other tools. At ten domains, that $99 base is $990 a month before any change in coverage. Price the setup you would actually report, then ask the vendor to show how its numbers were collected.

1. **Does it verify bot identity, or trust the User-Agent?** Ask which published IP ranges it checks and how it handles requests it cannot verify. Our `robots.txt` count was about four times its verified count.
2. **Does it separate retrieval agents from training crawlers?** Ask to see both columns on the same domain. Combining them makes answer-related activity look larger without showing more answer-page access.
3. **Can you afford the coverage you intend to sell?** Check domains, tracked prompts, engines, and the collection method at the tier you would buy. In the [six-tool comparison](https://aissist.io/insights/best-ai-visibility-tools), collection methods differed; a prompt-based observation is not the same measurement as an ingested server log.
4. **Does it execute fixes, or only list them?** This is the item I would not skip. A dashboard suggesting 3 to 5 fixes across each of 10 client domains leaves your team 30 to 50 tickets to write, ship, and verify.

Those tickets are the cost of treating reporting as the finished job. The numbers support a narrower buying decision: insist on verified identities, separate retrieval fetches, and path-level evidence, then find out who acts when a page earns none. **If a platform cannot demonstrate that on one domain, I will not roll its raw bot-hit chart across ten.**

## Frequently Asked Questions

### How many claimed AI bot hits actually verify against the vendor's published IP ranges?

On groas.com over a 30-day window, only 3,419 of 13,835 claimed AI bot hits on robots.txt verified when the source IP was checked against the named vendor's published ranges. That is roughly one in four, so any script writing a User-Agent string accounts for most of the rest.

### Does an unverified AI bot hit mean the traffic was spoofed?

No. Unverified means the log does not support the vendor identity printed in that row, such as when no published IP range was available to check. It is not proof of spoofing, but it is already a serious limit for an agency putting a vendor name in a client report.

### Can I trust a User-Agent string like GPTBot to identify an AI crawler in my server logs?

No. A User-Agent is useful for sorting requests but is not identity verification, because any script can set that text. Verification requires checking the source IP against the vendor's published ranges, and some vendors like OpenAI publish separate verification files for different agents.

### Why did the homepage show 9,202 AI bot hits but only 962 retrieval-agent fetches?

Those are three different counts. 9,202 is requests whose User-Agent named an AI bot; 2,128 is requests whose IP matched the published vendor range; 962 is verified requests from retrieval agents used to fetch content for answers, not training crawls. Each step answers a different question.

### Does a verified retrieval-agent fetch mean my page appeared in an AI answer?

No. A verified retrieval fetch is evidence of access, not proof of a citation. The edge log records requests to the site; it does not show the assistant's final response or whether the fetched page was displayed as a source.

### How concentrated was verified AI retrieval traffic across the site's pages?

About 98% of verified retrieval-agent fetches went to just three paths: robots.txt, the homepage, and one changelog post. Deep blog pages had verified shares of only 2% to 8%, and one post logged 340 claimed hits but just 8 verified ones.

### Do requests to robots.txt count as AI bots reading my pages for answers?

No. A fetch of robots.txt only shows an agent checked the site's rules; it does not show the assistant read anything from the site to answer a customer. That is why a 98% concentration figure including robots.txt must not be read as an answer-page share.

### What should an agency include in a client report on AI bot traffic?

Three connected views: verified fetches by engine, which tests identity; verified retrieval-agent fetches, kept separate from training crawls; and the paths receiving those fetches, which shows whether activity is spread across the site or concentrated. Each view should state what was checked and what remains unknown.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#webpage","url":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable","name":"Only 1 in 4 Claimed AI Bot Hits Verified: What 30 Days of Edge Logs Mean for Agency Reports","description":"In 30 days of groas edge logs, 25% of claimed AI bot hits on robots.txt verified. Here’s what agencies can defend in a client report—and what they can’t.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/beac49ac-e479-4c0c-90de-58244721019e.png"},"breadcrumb":{"@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#breadcrumb"},"dateModified":"2026-10-10T05:28:49.773Z","mainEntity":{"@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Only 1 in 4 Claimed AI Bot Hits Verified: What 30 Days of Edge Logs Mean for Agency Reports","item":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#article","url":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable","mainEntityOfPage":{"@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Only 1 in 4 Claimed AI Bot Hits Verified: What 30 Days of Edge Logs Mean for Agency Reports","description":"In 30 days of groas edge logs, 25% of claimed AI bot hits on robots.txt verified. Here’s what agencies can defend in a client report—and what they can’t.","datePublished":"2026-10-10T05:28:49.714Z","dateModified":"2026-10-10T05:28:49.773Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/beac49ac-e479-4c0c-90de-58244721019e.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"verified, hits, claimed, bot, edge, fetches, bot hits, edge logs","wordCount":1958,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/only-1-in-4-ai-bot-visits-is-verifiable#faq","mainEntity":[{"@type":"Question","name":"How many claimed AI bot hits actually verify against the vendor's published IP ranges?","acceptedAnswer":{"@type":"Answer","text":"On groas.com over a 30-day window, only 3,419 of 13,835 claimed AI bot hits on robots.txt verified when the source IP was checked against the named vendor's published ranges. That is roughly one in four, so any script writing a User-Agent string accounts for most of the rest."}},{"@type":"Question","name":"Does an unverified AI bot hit mean the traffic was spoofed?","acceptedAnswer":{"@type":"Answer","text":"No. Unverified means the log does not support the vendor identity printed in that row, such as when no published IP range was available to check. It is not proof of spoofing, but it is already a serious limit for an agency putting a vendor name in a client report."}},{"@type":"Question","name":"Can I trust a User-Agent string like GPTBot to identify an AI crawler in my server logs?","acceptedAnswer":{"@type":"Answer","text":"No. A User-Agent is useful for sorting requests but is not identity verification, because any script can set that text. Verification requires checking the source IP against the vendor's published ranges, and some vendors like OpenAI publish separate verification files for different agents."}},{"@type":"Question","name":"Why did the homepage show 9,202 AI bot hits but only 962 retrieval-agent fetches?","acceptedAnswer":{"@type":"Answer","text":"Those are three different counts. 9,202 is requests whose User-Agent named an AI bot; 2,128 is requests whose IP matched the published vendor range; 962 is verified requests from retrieval agents used to fetch content for answers, not training crawls. Each step answers a different question."}},{"@type":"Question","name":"Does a verified retrieval-agent fetch mean my page appeared in an AI answer?","acceptedAnswer":{"@type":"Answer","text":"No. A verified retrieval fetch is evidence of access, not proof of a citation. The edge log records requests to the site; it does not show the assistant's final response or whether the fetched page was displayed as a source."}},{"@type":"Question","name":"How concentrated was verified AI retrieval traffic across the site's pages?","acceptedAnswer":{"@type":"Answer","text":"About 98% of verified retrieval-agent fetches went to just three paths: robots.txt, the homepage, and one changelog post. Deep blog pages had verified shares of only 2% to 8%, and one post logged 340 claimed hits but just 8 verified ones."}},{"@type":"Question","name":"Do requests to robots.txt count as AI bots reading my pages for answers?","acceptedAnswer":{"@type":"Answer","text":"No. A fetch of robots.txt only shows an agent checked the site's rules; it does not show the assistant read anything from the site to answer a customer. That is why a 98% concentration figure including robots.txt must not be read as an answer-page share."}},{"@type":"Question","name":"What should an agency include in a client report on AI bot traffic?","acceptedAnswer":{"@type":"Answer","text":"Three connected views: verified fetches by engine, which tests identity; verified retrieval-agent fetches, kept separate from training crawls; and the paths receiving those fetches, which shows whether activity is spread across the site or concentrated. Each view should state what was checked and what remains unknown."}}]}]}
```