---
title: "50,075 AI Bot Hits. Only 2,703 Were On-Demand Fetches"
description: "Only 2,703 of the 50,075 AI bot hits in our 30-day edge logs came from verified on-demand agents."
url: "https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve"
image: "https://groas.com/media/blog/94abb6869f973cd922a4afdc082e961f368da196cf707a64bf4253247169741f.png"
published: "2026-10-07T05:30:06.299Z"
modified: "2026-10-07T05:30:06.299Z"
---

[AI For Google Ads](https://groas.com/category/ai-for-google-ads) · October 7, 2026 · 9 min read

# 50,075 AI Bot Hits. Only 2,703 Were On-Demand Fetches

[DavidFounder & CEO @ groas](https://groas.com/author/david)

![Clay doorman with a magnifier checks a huge queue all wearing 'HELLO I'M ChatGPT' badges, many in fake disguises; the few let in just read the door sign.](https://groas.com/media/blog/94abb6869f973cd922a4afdc082e961f368da196cf707a64bf4253247169741f.png)

Only **2,703 of the 50,075 AI bot hits** in our 30-day edge logs came from verified on-demand agents. If I put the larger number in a visibility report, I would be counting mostly requests I could not even attribute to an AI provider.

The first filter cut those 50,075 user-agent matches to **8,573 verified requests (17.1%)**. The second isolated requests from `ChatGPT-User` and `Claude-User`, the agents that fetch pages in response to user activity: **2,703 (5.4% of the raw total)**. That last number matters more than the headline count. It still does not tell us whether a particular page appeared in an answer, earned a citation, or sent a buyer to groas.

These figures come from [our 30-day log analysis](https://groas.com/post/we-logged-30-days-of-ai-bots-hitting-our), not from a universal benchmark. They show why a dashboard full of bot hits is a poor substitute for measuring which pages on-demand agents actually request.

## Three counts, three different claims

A user-agent string is a requester’s claim about its identity. It is not proof. We checked claimed AI bot requests against provider network information, including [published IP ranges](https://developers.openai.com/api/docs/bots) and verification records, then separated on-demand agents from other verified crawlers. The counts describe requests, not people, prompts, citations, or revenue.

| Traffic classification     | Requests | Share of raw matches | What the count supports                         |
| -------------------------- | -------- | -------------------- | ----------------------------------------------- |
| Raw AI user-agent matches  | 50,075   | 100.0%               | A client sent a matching string                 |
| Verified provider requests | 8,573    | 17.1%                | The request passed our provider checks          |
| Verified on-demand fetches | 2,703    | 5.4%                 | `ChatGPT-User` or `Claude-User` requested a URL |

The middle row includes more than live retrieval. Verified crawlers can fetch pages for other operational purposes. The bottom row is narrower, but **an on-demand fetch is not a confirmed citation**. Server logs show that an agent requested a resource; they do not show the answer a person saw.

That leaves **41,502 requests** that matched an AI bot string but did not pass our verification filter. Some may have been scrapers or probes. The logs do not justify assigning a motive to every one of them. The risk is straightforward enough without that story: as [practitioner log audits](https://hey-eye.com/blog/how-to-verify-real-ai-crawlers-vs-spoofed-bots-server-side/) and [crawler-spoofing accounts](https://www.reddit.com/r/webdev/comments/1ds2p5r/we_caught_400_fake_chatgpt_bots_in_two_months/) illustrate, a requester can claim to be `ChatGPT-User` or `ClaudeBot` in a header. Counting that header as verified visibility gives the requester control of your report.

![Traffic narrows from 50,075 AI user-agent matches to 8,573 verified requests and 2,703 on-demand fetches.](https://groas.com/media/blog/fab183a82d2f6c303cf81cdfa30805d4176fc8f8f39ec20e0f1523759e9bb5e2.png)

### The blog archive looked busy. Its articles told another story

The `/blog` root path collected **4,356 AI user-agent matches** during the same window. If you stop there, the archive looks popular with answer engines. Only **263 requests (6.0%)** passed our provider checks, and we recorded **zero on-demand fetches** to that path.

That does not mean the articles under `/blog` were invisible. The archive is a listing page, not the articles themselves. Nor can I identify all **4,093 unverified requests** as content harvesters from the count alone. What I can say is that archive traffic made a weak visibility metric: it mixed a large unverified total with no observed on-demand requests to the archive URL.

Our earlier breakdown of category pages showed the same reporting trap. Listing hubs drew thousands of claimed bot requests but no recorded answer-time retrievals. A category page can help people navigate. Its bot-hit count does not tell you whether an assistant retrieved the underlying article to address a specific question.

**Report the article paths separately from their directory pages.** Otherwise, a noisy archive can make a quiet content catalog look successful.

#### Three technical posts drew 518 on-demand fetches

When we grouped the **2,703 verified on-demand fetches** by path, three articles accounted for **518**, or **19.2%** of that total. They were practical, dated pieces rather than broad positioning copy.

| Article                | On-demand fetches | What the article covered                                                |
| ---------------------- | ----------------- | ----------------------------------------------------------------------- |
| YouTube 2026 guide     | 273               | Frequency caps, format adjustments, and policy updates                  |
| AI Max guide           | 139               | Parameter changes and automated budget-pacing thresholds                |
| Google Ads updates log | 106               | Bidding-target adjustments, limited ad serving, and setting retirements |
| **Three-post total**   | **518**           | **19.2% of all recorded on-demand fetches**                             |

These are fetch counts from our logs, not counts of answers that quoted the posts. We cannot recover the user’s exact question or the assistant’s final wording from a request alone. What the paths do show is **where on-demand agents went** when they requested material from our domain.

![On-demand fetches to three dated technical posts stand out from the other article paths.](https://groas.com/media/blog/6aa0cb5ba45c8a9bf28e85649231ac1136876f56a27860b657f14bbbc3481e78.png)

As we noted when looking at [what those pages had in common](https://groas.com/post/what-30-days-of-ai-crawler-logs-say-abou), each supplied details that are useful to check against a current source: dated changes, specific settings, and direct answers. They put the operational point near the top instead of making a reader wade through a ceremonial introduction to discover it.

I would not turn three posts into a rule that every generic explainer fails or every table wins. The logs do not run that experiment. They do give me a clear editorial bet: when a page covers a changing technical detail, **make the detail easy to find and verify**. If a team calls that answer engine optimization, fine. I call it writing down the answer.

#### Most on-demand requests did not land on an article

Two paths took **2,107 of the 2,703 on-demand fetches**, nearly **78%** of the total: `/robots.txt` and the homepage. That changes how I read the three-post result. The articles won a meaningful share of recorded on-demand activity, but much of the denominator was infrastructure or navigation, not article retrieval.

| Path          | Agent          | On-demand requests | What we can observe                      |
| ------------- | -------------- | ------------------ | ---------------------------------------- |
| `/robots.txt` | `Claude-User`  | 1,160              | Repeated requests for crawl instructions |
| Homepage      | `ChatGPT-User` | 947                | Repeated requests for the root page      |
| **Combined**  |                | **2,107**          | **Nearly 78% of on-demand fetches**      |

[Anthropic’s crawler documentation](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) explains why crawl permissions matter. Our logs show frequent `Claude-User` requests for `/robots.txt`; they do not prove that every request preceded an article fetch or that a slow response cost us a citation. Still, if that file is broken or your CDN blocks the agent, fixing access is more useful than celebrating a rise in claimed bot traffic.

The **947 `ChatGPT-User` homepage requests** make the root page worth checking too. They do not reveal the prompts behind those requests. They do tell us the homepage was a frequent destination, so I want its primary content readable in the HTML an agent receives. A polished browser rendering is not much help if the initial response is an empty shell.

**Before optimizing a page for retrieval, confirm that the agent can read it.** That is less glamorous than a visibility score and considerably easier to test.

#### The limit: logs cannot settle Google visibility

This analysis covers what we could distinguish in our edge logs, chiefly verified requests from OpenAI and Anthropic agents. It does not produce a complete count of AI answers involving our site.

Google is the obvious blind spot. Its [crawler overview](https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers) and [AI features documentation](https://developers.google.com/search/docs/crawling-indexing/ai-features-and-your-website) describe an indexing setup that does not give us a separate, identifiable live AI Overview fetch in these logs. `Google-Extended` is a robots.txt control, not a request user agent we can count. A regular `Googlebot` visit therefore cannot be labelled an AI Overview appearance just because we would like another row in the report.

The same restraint applies to the **2,703 on-demand fetches**. We can identify the requesting agent and path, then investigate whether the page was accessible. We cannot turn that into a verified citation or a qualified visit. To examine what an assistant displays for relevant questions, we need prompt evaluations alongside the logs; to assess business value, we need to look beyond fetches altogether.

**Use logs to measure access and retrieval, not to claim exposure they cannot show.**

#### Run the three-number audit on your own site

Here is the test I would run before accepting a report that says AI visibility surged because bot hits went up. The question is narrow: _How much of the claimed bot traffic can we verify, and which paths do on-demand agents request?_ Run it prospectively on one **30-day** window. Keep the window, path handling, and verification rules consistent across all three counts so the filters, not a changing setup, explain the drop.

![Three-stage audit: match AI user agents, verify provider requests, then isolate on-demand agents by path.](https://groas.com/media/blog/dffede953011ac08940d4e7fc3e56cf251255b7e683849150d99b95f01d65796.png)

1. **Metric A — raw matches.** Pull edge or access logs from your CDN or server and count requests whose `User-Agent` contains the identifiers you intend to track, such as `ChatGPT-User`, `GPTBot`, `Claude-User`, `ClaudeBot`, and `PerplexityBot`. Keep the request path. This is your noisy starting count, not your result.
2. **Metric B — verified requests.** Check the source network against the relevant provider’s [published information](https://developers.openai.com/api/docs/bots) and, where appropriate, use [forward-confirmed reverse DNS](https://datadome.co/bot-management-protection/oai-adsbot-crawler/). Record what passes your checks and what does not. A failed check means _unverified by this method_, not automatically malicious. If crawler access itself is in doubt, our [curl and CDN audit](https://groas.com/post/can-chatgpt-actually-read-your-site-a-45) is a useful preliminary check.
3. **Metric C — on-demand fetches.** Within the verified set, count requests from `ChatGPT-User` and `Claude-User`. Group them by path, separating `/robots.txt`, the homepage, archives, and individual content pages. Do not label the resulting count “citations.”

Calculate **Metric B / Metric A** to see how much the raw headline survives verification. Then inspect Metric C’s path distribution rather than treating every fetch as a content win. On our site, those calculations produced **17.1% verified requests** and **2,703 on-demand fetches**, with `/robots.txt` and the homepage taking nearly **78%** of the latter. Those are comparisons for our window, not thresholds that diagnose every site.

I expect many raw dashboards to shrink under this test. The mechanism is mundane: user-agent matching accepts a self-reported string, while provider checks ask whether the network identity supports it. I also expect the path breakdown to complicate the remaining headline, because an on-demand request for a permission file is not the same thing as a request for an article. If your verified share stays high and on-demand fetches reach a broad set of useful pages, keep that evidence. If the count collapses or clusters on infrastructure, change the report before you change the content strategy.

#### Put the verified number in the report

I would make three changes from these logs. First, stop presenting raw AI bot matches as AI visibility. Show the raw count, the verified count, and the on-demand count together, with the verification method stated plainly. A percentage increase in unverified strings is not evidence that more buyers saw your brand.

Second, check the paths agents request. Make sure crawl permissions work and that the homepage and important articles return readable content. Our `/robots.txt` and homepage counts make those checks hard to dismiss. They do not justify a made-up timeout rule or a promise that one server tweak will earn a citation.

Third, put editorial effort into pages that answer specific, changing questions clearly. Our three technical posts drew **518 on-demand fetches**; that is a reason to examine their structure and produce more useful reference material, not a guarantee that copying their tables will reproduce the count. Measure future pages by their verified on-demand fetches _per page_, then check displayed answers and business outcomes separately.

At groas, I would rather explain a smaller number tied to a real mechanism than sell a spectacular one built from a header anyone can type. The decision these **50,075** matches support is not “publish more for bots.” It is **verify the requester, inspect the requested page, and report only what the evidence can carry**.

## Frequently Asked Questions

### Do all AI bot hits in server logs come from real AI providers?

No. Of 50,075 requests whose user-agent string matched an AI bot in groas's 30-day edge logs, only 8,573 (17.1%) passed checks against provider network information such as published IP ranges. The remaining 41,502 could not be verified, because a requester can type any agent name, including ChatGPT-User, into a header.

### What is the difference between raw AI bot hits, verified requests, and on-demand fetches?

Raw matches count any request whose user-agent string looks like an AI bot, verified requests are those confirmed against provider network information, and on-demand fetches are the subset from agents that retrieve pages in response to user activity, such as ChatGPT-User and Claude-User. In groas's 30-day logs, these counts were 50,075, 8,573, and 2,703 respectively.

### Does an on-demand fetch mean the page was cited in an AI answer?

No. Server logs show only that an agent requested a resource; they do not reveal the answer a person saw, whether the page was cited, or whether it sent a buyer. To assess what assistants actually display, you need prompt evaluations alongside the logs.

### Can the /blog archive's bot-hit count show how visible my articles are to answer engines?

No. During the same 30-day window, groas's /blog root path collected 4,356 AI user-agent matches, but only 263 (6.0%) passed provider checks and zero were on-demand fetches. The archive is a listing page, not the articles themselves, so report article paths separately from directory pages.

### What kind of content do on-demand AI agents fetch most often?

In groas's logs, three dated technical posts accounted for 518 of the 2,703 verified on-demand fetches, or 19.2%: a YouTube 2026 guide (273), an AI Max guide (139), and a Google Ads updates log (106). Each supplied dated changes, specific settings, and direct answers placed near the top of the page. That is an editorial bet, not a proven rule.

### What should I check before optimizing a page for AI retrieval?

Confirm the agent can read it first. Nearly 78% of groas's 2,703 on-demand fetches went to /robots.txt (1,160 requests from Claude-User) and the homepage (947 requests from ChatGPT-User). Make sure crawl permissions work and that the homepage and key articles return readable content in the initial HTML rather than an empty shell.

### Can server logs tell me if my site appears in Google's AI Overviews?

No. Google's indexing setup does not produce a separate, identifiable live AI Overview fetch in these logs, and Google-Extended is a robots.txt control rather than a countable request user agent. A regular Googlebot visit cannot be labelled an AI Overview appearance, so logs should be used to measure access and retrieval, not exposure they cannot show.

### How can I audit my own AI bot traffic before trusting a visibility report?

Run a three-number audit over one 30-day window with consistent rules: count raw user-agent matches (Metric A), verify requesters against provider information such as published IP ranges or forward-confirmed reverse DNS (Metric B), then count ChatGPT-User and Claude-User fetches grouped by path (Metric C). Calculate B/A to see how much the headline survives verification, and inspect Metric C's path distribution without labeling fetches as citations.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#webpage","url":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve","name":"50,075 AI Bot Hits. Only 2,703 Were On-Demand Fetches","description":"Only 2,703 of the 50,075 AI bot hits in our 30-day edge logs came from verified on-demand agents.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://groas.com/media/blog/94abb6869f973cd922a4afdc082e961f368da196cf707a64bf4253247169741f.png"},"breadcrumb":{"@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#breadcrumb"},"dateModified":"2026-10-07T05:30:06.299Z","mainEntity":{"@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"AI For Google Ads","item":"https://groas.com/category/ai-for-google-ads"},{"@type":"ListItem","position":4,"name":"50,075 AI Bot Hits. Only 2,703 Were On-Demand Fetches","item":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#article","url":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve","mainEntityOfPage":{"@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"50,075 AI Bot Hits. Only 2,703 Were On-Demand Fetches","description":"Only 2,703 of the 50,075 AI bot hits in our 30-day edge logs came from verified on-demand agents. If I put the larger number in a visibility report, I would be counting mostly requests I could not even attribute to an AI provider. The fi…","datePublished":"2026-10-07T05:30:06.299Z","dateModified":"2026-10-07T05:30:06.299Z","image":{"@type":"ImageObject","url":"https://groas.com/media/blog/94abb6869f973cd922a4afdc082e961f368da196cf707a64bf4253247169741f.png"},"author":{"@id":"https://groas.com/author/david#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"articleSection":"AI For Google Ads","keywords":"on-demand, requests, bot, bot hits, on-demand fetches, 703, logs, hits","wordCount":1863,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/david#person","name":"David","jobTitle":"Founder & CEO @ groas","image":"https://groas.com/media/blog/363be9ad5255816654dac065d01bf1c6063110b400bb38cc199570267232027c.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"FAQPage","@id":"https://groas.com/post/50-075-ai-bot-hits-in-30-days-17-were-ve#faq","mainEntity":[{"@type":"Question","name":"Do all AI bot hits in server logs come from real AI providers?","acceptedAnswer":{"@type":"Answer","text":"No. Of 50,075 requests whose user-agent string matched an AI bot in groas's 30-day edge logs, only 8,573 (17.1%) passed checks against provider network information such as published IP ranges. The remaining 41,502 could not be verified, because a requester can type any agent name, including ChatGPT-User, into a header."}},{"@type":"Question","name":"What is the difference between raw AI bot hits, verified requests, and on-demand fetches?","acceptedAnswer":{"@type":"Answer","text":"Raw matches count any request whose user-agent string looks like an AI bot, verified requests are those confirmed against provider network information, and on-demand fetches are the subset from agents that retrieve pages in response to user activity, such as ChatGPT-User and Claude-User. In groas's 30-day logs, these counts were 50,075, 8,573, and 2,703 respectively."}},{"@type":"Question","name":"Does an on-demand fetch mean the page was cited in an AI answer?","acceptedAnswer":{"@type":"Answer","text":"No. Server logs show only that an agent requested a resource; they do not reveal the answer a person saw, whether the page was cited, or whether it sent a buyer. To assess what assistants actually display, you need prompt evaluations alongside the logs."}},{"@type":"Question","name":"Can the /blog archive's bot-hit count show how visible my articles are to answer engines?","acceptedAnswer":{"@type":"Answer","text":"No. During the same 30-day window, groas's /blog root path collected 4,356 AI user-agent matches, but only 263 (6.0%) passed provider checks and zero were on-demand fetches. The archive is a listing page, not the articles themselves, so report article paths separately from directory pages."}},{"@type":"Question","name":"What kind of content do on-demand AI agents fetch most often?","acceptedAnswer":{"@type":"Answer","text":"In groas's logs, three dated technical posts accounted for 518 of the 2,703 verified on-demand fetches, or 19.2%: a YouTube 2026 guide (273), an AI Max guide (139), and a Google Ads updates log (106). Each supplied dated changes, specific settings, and direct answers placed near the top of the page. That is an editorial bet, not a proven rule."}},{"@type":"Question","name":"What should I check before optimizing a page for AI retrieval?","acceptedAnswer":{"@type":"Answer","text":"Confirm the agent can read it first. Nearly 78% of groas's 2,703 on-demand fetches went to /robots.txt (1,160 requests from Claude-User) and the homepage (947 requests from ChatGPT-User). Make sure crawl permissions work and that the homepage and key articles return readable content in the initial HTML rather than an empty shell."}},{"@type":"Question","name":"Can server logs tell me if my site appears in Google's AI Overviews?","acceptedAnswer":{"@type":"Answer","text":"No. Google's indexing setup does not produce a separate, identifiable live AI Overview fetch in these logs, and Google-Extended is a robots.txt control rather than a countable request user agent. A regular Googlebot visit cannot be labelled an AI Overview appearance, so logs should be used to measure access and retrieval, not exposure they cannot show."}},{"@type":"Question","name":"How can I audit my own AI bot traffic before trusting a visibility report?","acceptedAnswer":{"@type":"Answer","text":"Run a three-number audit over one 30-day window with consistent rules: count raw user-agent matches (Metric A), verify requesters against provider information such as published IP ranges or forward-confirmed reverse DNS (Metric B), then count ChatGPT-User and Claude-User fetches grouped by path (Metric C). Calculate B/A to see how much the headline survives verification, and inspect Metric C's path distribution without labeling fetches as citations."}}]}]}
```