Only 1 in 4 Claimed AI Bot Hits Verified: What 30 Days of Edge Logs Mean for Agency Reports

On kindredworkshop.com, only 3,419 of 13,835 claimed AI bot hits on robots.txt verified over 30 days. That is roughly one in four. If an agency reports the other three as visits from named AI vendors, it is asking the client to trust a User-Agent string that any script can write.

The problem follows the report down the site. About 23% of claimed homepage hits verified; deep blog pages ranged from 2% to 8%. And roughly 98% of verified retrieval-agent fetches went to just three paths. Those figures do not tell me which pages appeared in answers. They do tell me that a raw bot-hits chart is a poor account of what happened.

Start with the claim the edge log can test

This is our own 30 days of Cloudflare edge data for kindredworkshop.com, not a benchmark for every site. I logged requests claiming an AI bot User-Agent and checked the source IP against the named vendor’s published ranges. A match counted as verified. I did not use reverse DNS as a fallback; when I had no published range to check, I marked the request unverifiable, not verified.

That distinction matters. Unverified does not mean proven spoofed. It means the log does not support the vendor identity printed in that row. For an agency putting a vendor name in a client report, that is already a serious limit.

Source: kindredworkshop.com Cloudflare edge logs, 30-day window; User-Agent claims checked against published vendor IP ranges. Percentages are verified hits divided by claimed hits on the same path. The retrieval-agent column is a narrower count, not a count of displayed answers.

A User-Agent is useful for sorting requests. It is not identity verification. OpenAI, for example, publishes separate verification files for three agents, so even a familiar bot name requires the right check. I used to give the string more credit when scanning logs. I would not put it in a client-facing AI visibility total without checking the IP now.

Other studies illustrate why the distinction is worth making, but they are not direct comparisons with our percentages. A two-week study of 16 AI crawlers found 5.7% of traffic bearing an AI-crawler User-Agent was spoofed, rising to 16.7% for ChatGPT-User. On one small unpromoted domain, only 6 of 33 claimed live assistant fetches verified. Different sites and methods produce different numbers. My edge log supports the narrower conclusion: most claimed hits in these groas path counts could not be verified as the vendor they named.

The homepage has three numbers, not one

The homepage received 9,202 claimed hits. IP checks brought that down to 2,128 verified hits. Filtering for retrieval agents brought it down again to 962 fetches. Each step answers a different question; none is a substitute for the next.

• 9,202 claimed hits: requests whose User-Agent named an AI bot.
• 2,128 verified hits: requests whose IP matched the relevant published vendor range.
• 962 retrieval-agent fetches: verified homepage requests from agents used to retrieve content for answers, rather than training crawls.

Retrieval agents such as OAI-SearchBot and ChatGPT-User serve a different role from training crawlers such as GPTBot. If I combine the two in a single answer-visibility number, a crawl that cannot itself produce a citation makes the report look better. The split is more useful than the bigger total.

Still, I would not tell a client that 962 homepage fetches produced 962 answers, prompts, or citations. The edge log records requests to our site. It does not show the assistant’s final response or tell us whether the fetched page was displayed as a source. A verified retrieval fetch is evidence of access, not proof of a citation. That is the number I would report, with that limit beside it.

Deep-page counts shrink further after verification

The homepage was not the worst case. In the same 30 days, our category-style pages each drew around 3,000 claimed hits, but their verified share sat near 7%. They recorded no retrieval-agent fetches in the count used here. One deep post, /post/flat-rate-aeo-pricing-per-domain, logged 340 claimed hits and 8 verified: 2.4%. Across deep blog pages, the verified share ranged from 2% to 8%.

Those figures come from the same kindredworkshop.com edge-log checks as the homepage table. The range matters more than a single blended average: it shows how little a raw page-level hit count tells me about a particular URL. A dashboard could give that post a bar for 340 ‘AI visits.’ I could substantiate 8 vendor-IP matches. I would not quietly rename the remaining 332 as visits from that vendor, nor claim I had established what each unverified request was.

There is a reason to be cautious about odd traffic on deep URLs. On a small unpromoted test domain, claimed ChatGPT-User requests went looking for .env.production and secrets.yaml rather than useful pages. The study of 16 AI crawlers describes spoofing patterns that make a familiar User-Agent and a quick glance at a log insufficient checks. Neither source proves that every unverified request in our logs was an attacker. It does explain why I refuse to treat a claimed name as a verified reader.

The practical decision is simple: verify before ranking pages by AI bot traffic. Otherwise the pages with the busiest unverified logs can look like the biggest visibility wins.

Three paths account for nearly all retrieval fetches

After IP verification and the retrieval-agent filter, about 98% of fetches in our 30-day edge data went to three paths: robots.txt, the homepage, and one changelog post. The homepage recorded 962 retrieval-agent fetches and the changelog post 232. The other 197 site paths shared roughly 2% of the verified retrieval-agent fetches.

Source: the same kindredworkshop.com 30-day edge logs, grouped by path after IP and agent-type checks. The 98% figure includes requests to robots.txt for discovery; it must not be read as the share of answer-page fetches, much less the share of citations.

That last qualification is not fine print. A fetch of robots.txt tells me an agent checked the site’s rules. It does not tell me the assistant read that file to answer a customer. Even the homepage and changelog counts stop at the request. I can say retrieval agents reached those pages; I cannot use edge logs alone to say how often the pages appeared in responses.

I used to tell clients that coverage meant getting every page cited. I was wrong to make that the starting promise. On this site, the measurable activity was highly concentrated, and a long list of URLs with claimed bot hits would have hidden that fact. Name the paths that earned verified retrieval fetches first. Make the case for work on the remaining pages with evidence, not with a fuller-looking chart.

Why the standard dashboard overstates what it knows

The failure starts at collection. A script can set its User-Agent to GPTBot or PerplexityBot; the receiving server will log that text. To support the identity claim, a report needs a further check against the vendor’s published ranges, with any additional vendor-specific verification kept distinct. Skip that step and the chart measures claims, including claims the agency cannot substantiate.

It can also mix incompatible activity. A training crawl is not a retrieval fetch, a retrieval fetch is not a displayed citation, and a displayed citation is not attributable revenue. Each may matter for a different question. None should borrow the label of the next just because the graph looks sparse otherwise.

Some activity is also outside what this method can separate. As this account of bot-log limits explains, Googlebot crawling and the Google-Extended permission flag do not give an edge-log reader a separate Gemini fetcher to count. I would rather leave that gap visible than fill it with an invented ‘Gemini visits’ total. A test of top domains found two in three blocked none of ten test bots, another reminder that site-level bot controls and bot-identity reporting are different jobs.

So I do not want a grand total called AI visits. I want a table whose columns say what was checked and what remains unknown. It is less theatrical. It is also a report a client can challenge without making the whole thing fall over.

Put these figures in the client report

For a client, I would show three connected views: verified fetches by engine, verified retrieval-agent fetches, and the paths receiving those fetches. The first tests identity. The second separates agents that retrieve content for answers from training crawlers. The third shows whether activity is spread across the site or concentrated somewhere the team can inspect.

The groas counts come from the 30-day edge-log window above. An agency tracker comparison also discusses engine breakdowns and cited-page analysis, but a cited-page measure requires evidence beyond an edge request.

For a client asking why a 200-path site has activity concentrated in a few places, this table is more helpful than a promise of blanket coverage. It shows where the requests went and where the evidence stops. To run the checks on a client domain, start with the curl and CDN audit, then use the edge-log tutorial for the IP-verification method. Neither step turns a server request into a citation; that boundary belongs in the report too.

Before you roll a platform across ten domains

You are choosing a reporting standard you will repeat across clients, not just a dashboard your team likes in a demo. Run this checklist on a trial domain first. The pricing question deserves the same scrutiny as the data: the AEO tools comparison lists a $99-per-domain, 25-prompt base for Semrush and tier or add-on limits for other tools. At ten domains, that $99 base is $990 a month before any change in coverage. Price the setup you would actually report, then ask the vendor to show how its numbers were collected.

1. Does it verify bot identity, or trust the User-Agent? Ask which published IP ranges it checks and how it handles requests it cannot verify. Our robots.txt count was about four times its verified count.
2. Does it separate retrieval agents from training crawlers? Ask to see both columns on the same domain. Combining them makes answer-related activity look larger without showing more answer-page access.
3. Can you afford the coverage you intend to sell? Check domains, tracked prompts, engines, and the collection method at the tier you would buy. In the six-tool comparison, collection methods differed; a prompt-based observation is not the same measurement as an ingested server log.
4. Does it execute fixes, or only list them? This is the item I would not skip. A dashboard suggesting 3 to 5 fixes across each of 10 client domains leaves your team 30 to 50 tickets to write, ship, and verify.

Those tickets are the cost of treating reporting as the finished job. The numbers support a narrower buying decision: insist on verified identities, separate retrieval fetches, and path-level evidence, then find out who acts when a page earns none. If a platform cannot demonstrate that on one domain, I will not roll its raw bot-hit chart across ten.