---
title: "Build Your Own Per-Prompt AI Search Tracker for ChatGPT, Gemini, and AI Overviews"
description: "Build a 25-prompt tracker that logs mentions, citations, and competitors by AI search engine, then checks each site change against weekly results."
url: "https://groas.com/post/build-a-per-prompt-ai-search-tracker-in"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca4a57c7-3d3c-485d-b064-a07899837189.png"
published: "2026-10-09T05:19:57.541Z"
modified: "2026-10-09T05:19:57.602Z"
---

October 9, 2026 · 9 min read

# Build Your Own Per-Prompt AI Search Tracker for ChatGPT, Gemini, and AI Overviews

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A blender of featureless beige purée beside a tray of 25 sorted whole fruits, some slots empty: a blended score versus a per-question log.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca4a57c7-3d3c-485d-b064-a07899837189.png)

In this article

1. [What you will have, and what you need](#what-you-will-have-and-what-you-need)
2. [Step 1: Freeze 25 questions buyers actually ask](#step-1-freeze-25-questions-buyers-actually-ask)
3. [Step 2: Collect three grounded answers per engine](#step-2-collect-three-grounded-answers-per-engine)
4. [Step 3: Keep mentions, citations, and recommendations apart](#step-3-keep-mentions-citations-and-recommendations-apart)
5. [Step 4: Put the collection on a weekly schedule](#step-4-put-the-collection-on-a-weekly-schedule)
6. [Step 5: Tag each site change to the prompts it targets](#step-5-tag-each-site-change-to-the-prompts-it-targets)
7. [Verify the tracker, then ship the first fix](#verify-the-tracker-then-ship-the-first-fix)
8. [When to stop building and start buying](#when-to-stop-building-and-start-buying)

A blended AI visibility score cannot tell you which buyer question sent a citation to your competitor. **A per-prompt, per-engine log can.** In [_Your AI Visibility Score Is Impression Share With a New Haircut_](https://groas.com/post/your-ai-visibility-score-is-impression-s), I made the case against buying the score. The obvious follow-up is what to build instead. This is the hands-on answer: a small tracker that records who got named and cited, then lets you check what happened after you changed a page.

## What you will have, and what you need

At the end, you will have three linked tabs in a Google Sheet or Airtable base:

- **`raw_runs`:** One row per prompt, engine, and run. It records the timestamp, model version, brand mention, brand citation, recommendation, competitors, and exact cited URLs.
- **A prompt rollup:** Mention and citation counts across three runs per prompt, kept separate for ChatGPT, Gemini, and Google AI Overviews. It shows where you appeared and where a competitor appeared instead.
- **`change_log`:** A dated record of site edits, title revisions, and structured data updates, tagged to the prompts each change targets.

Before starting, get edit access to a fresh sheet or base; OpenAI and Google AI Studio API keys; a SERP API account such as [SerpApi](https://serpapi.com/blog/ai-overview-rank-tracker-using-python/) or DataForSEO; and a runner such as [n8n](https://n8n.io/workflows/16903-track-brand-visibility-in-chatgpt-results-with-openai-and-google-sheets/), Make, or a scheduled Python script. API calls and SERP requests have costs. Check your own usage rather than treating a dashboard subscription price as the only alternative.

## Step 1: Freeze 25 questions buyers actually ask

Open your CRM or call recording software, export the last 90 days of support tickets, and pull your Google Search Console query report. Look for comparison and constraint language: `vs`, `alternative`, `budget`, and `how to choose`. Buyers bring an LLM messy questions, not just clipped keywords. Use their phrasing, particularly sales objections and unstructured form fields, to assemble your panel. [J.M. Redman’s method for building AI query lists](https://www.jmredman.com/blog/how-to-build-an-ai-search-query-tracking-list) is useful here; the point is to [measure visibility against buyer questions](https://groas.com/post/how-to-measure-your-brand-s-visibility-i-2), not questions you wrote to flatter your brand.

Create a tab named `prompt_bank` with these columns. The rows below show the format; replace the examples with your buyers’ questions.

```csv
prompt_id,category,intent_stage,prompt_text
P01,pricing,evaluation,"What is the most reliable inventory tool for Shopify under $300 a month?"
P02,comparison,decision,"Competitor A vs Competitor B for a 15-person remote team: pros and cons"
P03,migration,consideration,"How painful is it to switch from Competitor A to an open-source alternative?"
P04,feature_fit,evaluation,"Best warehouse management software that integrates with QuickBooks Online"
P05,reputation,decision,"Is [Your Brand] reliable for enterprise reporting or only for SMBs?"
```

**Action:** Fill `P01` through `P25`. Classify each as problem research, vendor comparison, or technical evaluation, and use `category` for a narrower label. Put your brand name in no more than five prompts. Freeze the wording for at least 30 days.

**Expected result:** A stable panel of 25 unique IDs, with at least 20 unbranded questions where you want to be discovered on merit.

**Common mistake:** Filling the bank with `What does [Your Brand] do?` and similar softballs. A high mention rate on those prompts says little about whether an uncommitted buyer finds you.

## Step 2: Collect three grounded answers per engine

**One answer is a snapshot, not a baseline.** Sources and wording can change between runs of the same prompt. Query each prompt three times on each engine and keep the responses separate. That gives you a frequency to inspect instead of asking one Tuesday morning answer to stand in for the whole market.

![Three runs of each prompt across ChatGPT, Gemini, and Google AI Overviews feeding a structured run log](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/1399744c-e060-468d-aaf5-1f2b7199bbc2.png)

For ChatGPT and Gemini, configure API requests with live web search grounding. For Gemini, enable the `google_search` tool on a model that supports it; record the exact model ID you use. For Google AI Overviews, request a Google search result through your SERP API and inspect its `ai_overview` object. This Python function shows the SerpApi portion:

```python
import requests

def fetch_google_aio(prompt_text: str, api_key: str):
    params = {
        "engine": "google",
        "q": prompt_text,
        "api_key": api_key,
        "gl": "us",
        "hl": "en",
    }
    res = requests.get("https://serpapi.com/search", params=params)
    res.raise_for_status()
    aio = res.json().get("ai_overview")
    if not aio:
        return {"has_aio": False, "citations": [], "text": ""}

    references = aio.get("references", [])
    cited_urls = [ref.get("link") for ref in references if ref.get("link")]
    return {
        "has_aio": True,
        "citations": cited_urls,
        "text": aio.get("text", ""),
    }
```

**Action:** Loop through `prompt_bank` for `run_1`, `run_2`, and `run_3` on each engine. Save the response text, exact external URLs, timestamp, engine, model version where applicable, and whether an AI Overview appeared. Inspect your provider’s returned Overview fields before relying on the `text` and `references` extraction shown above.

**Expected result:** For 25 prompts, collect 75 records per engine: 225 observations per cycle. A Google result without an AI Overview remains a recorded observation, marked `has_aio: False`; it is not evidence that an Overview rejected your brand.

**Common mistake:** Sending an ungrounded chat request and calling its answer a live search result. Check that retrieval is enabled before you interpret the output.

## Step 3: Keep mentions, citations, and recommendations apart

A brand named in passing did not necessarily earn a link or a recommendation. **Score those as three different events.** Also retain the answers where you were absent; without them, your mention percentage has no denominator. [Konabayev’s citation schema](https://konabayev.com/blog/ai-citation-tracking-template/) separates mentions, links, and absences, while this [spreadsheet approach to AI visibility](https://bing.ly/blog/track-ai-visibility-in-google-sheets) emphasizes recording the competitor domains that appeared instead.

![The three separate AI presence checks: mentioned, cited, and recommended](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/07b5ad83-0bb9-41f1-9005-00cddf894bd1.png)

Route each raw answer and its extracted URLs through a lightweight evaluation model such as `gpt-4o-mini` at `temperature: 0`. Give it the brand name and competitor names you want checked. Require JSON with this structure; keep the underlying response so you can audit a questionable classification.

```json
{
  "brand_mentioned": true,
  "brand_cited_url": true,
  "brand_recommended": false,
  "rank_position": 2,
  "competitors_named": ["Competitor A", "Competitor B"],
  "cited_external_domains": ["g2.com", "capterra.com", "competitor-a.com"],
  "recommendation_context": "Competitor A was the primary pick; [Your Brand] was noted as an enterprise option."
}
```

**Action:** Write one row for every observation to `raw_runs`. Include `prompt_id`, `run_date`, `engine`, `model_version`, `run_number`, `has_aio`, `brand_mentioned`, `brand_cited`, `brand_recommended`, `rank_position`, `competitors_named`, `cited_domains`, and `cited_urls`. Store the three brand flags as `0` or `1`. Preserve the raw answer alongside the scored record.

**Expected result:** Exactly 225 rows for a complete cycle. In your rollup, calculate mention and citation frequency from the three rows for each prompt and engine, and show the competing domains beside them. Keep `has_aio: False` rows visible rather than folding them into an AI Overview citation rate.

**Common mistake:** Writing a row only when your brand appears. Twelve positive rows without the negative rows are twelve anecdotes, not a coverage rate. Another mistake is letting a classifier infer a cited domain from answer text when the extracted URL list does not contain it. Spot-check both.

## Step 4: Put the collection on a weekly schedule

Manual API-console queries are fine while you test the schema. They are a poor recurring process. Deploy the runner in n8n, Make, or Python on GitHub Actions or a virtual private server. Schedule one cycle per week; `Tuesday 06:00 UTC` is a workable setting. Keep the collection schedule fixed so a shift in timing does not masquerade as a change in visibility.

Have the runner read `prompt_bank`, call each engine three times per prompt, pause three seconds between repeated calls, score the responses, and append the rows to `raw_runs`. Set an automatic retry policy of three attempts with exponential backoff for network drops. If a request still fails, log the failure rather than manufacturing a zero.

Spend ten minutes on a weekly spot-check. Review records where `brand_mentioned` is true but `rank_position` is blank, and read any answer that mentions your brand while warning against it. A model can get the name right and the meaning wrong.

**Action:** Set the weekly schedule, retries, pause, and append operation. Run one complete cycle before leaving it unattended.

**Expected result:** The sheet receives 225 fresh observations on schedule, or shows a visible failure you can investigate. Mention, citation, and competitor columns populate without hand-copying answers.

**Common mistake:** Polling several times a day, then treating every source shuffle as an urgent trend. More rows do not fix a noisy interpretation; they can just make the alarm louder.

![Paper change log beside a terminal displaying JSON output](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4082acb4-0e9b-4a37-bc07-91c5bc2bae18.png)

## Step 5: Tag each site change to the prompts it targets

The run log tells you what happened in answers. It does not tell you what your team shipped. Create `change_log` before you start revising pages, or you will be reconstructing dates from memory later. Use the target prompt IDs to connect a pricing edit to pricing questions, not to every movement in the sheet. A [dated change log for AI search work](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGdyNApLLOHkptzwHI4wPYGlHeO8O7dZAWJ64zOe28vOAL_2ha0dtAIuwX10ulnKz5xcv-5Fv740eXXuc4cSvdKOTKwVTXXHzeU8sW-Vyk0wTSUQ5xJ1j7oZ5l0PaCoEi7_MILvs0DzK1TKF4YgNMfnrdy3P9Na8mug=) makes a before-and-after check possible. It does not turn that check into proof of causation.

```csv
change_id,date_shipped,target_url,change_type,target_prompt_ids,hypothesis
C01,2026-10-12,"/pricing","structured_table","P01,P04","A clear pricing table may help an engine extract the relevant tiers."
C02,2026-10-15,"/vs-competitor-a","direct_answer_h2","P02,P03","A factual comparison table may make the page easier to cite for migration prompts."
```

**Action:** Within 24 hours of an edit going live, log its date, URL, type, target prompt IDs, and hypothesis. When you have enough history, compare the targeted prompts for four weeks before and four weeks after the change, engine by engine. Look at competitor citations as well as your own.

**Expected result:** A ledger that lets you say whether your targeted prompt cohort moved after a specific edit, while still seeing whether unrelated prompts moved too.

**Common mistake:** Writing `updated SEO` with no URL or prompt IDs. That note will not help you distinguish a deliberate fix from ordinary answer variation.

## Verify the tracker, then ship the first fix

Open `raw_runs` after the scheduled collection. Filter for the latest `run_date` and check the following in order:

1. **Count 225 rows** for a complete 25-prompt, three-engine, three-run cycle. Investigate missing rows as failures, not brand absences.
2. Check that every completed response has a `brand_mentioned` value, and inspect cited URLs and domains against the raw payload. For Google, separate `has_aio: False` from an Overview that appeared without citing you.
3. Open the rollup. Confirm that each frequency comes from three runs of the same prompt on the same engine, not a blended count across engines.
4. Check `change_log` for the URL, ship date, and prompt IDs of any edit you plan to evaluate.

Then find a high-priority unbranded question where you appeared in zero of three runs and a direct competitor was cited in all three. Open the captured competitor URL. Inspect how it answers the buyer’s question: a clear heading, a concise answer, or a comparison table may make the relevant information easier to extract. Revise your own target page for the same clarity, without copying the competitor’s claims. Log the edit as `C03`, then watch that prompt and engine over the next four weekly runs. **Change the page first; watch the exact prompt next.**

## When to stop building and start buying

This tracker is cheap to run compared with the operator time it can consume. API collectors need maintenance, grounding settings can change, and classifications need spot-checks. If your team is spending hours debugging requests and still more hours interpreting rows, the spreadsheet is not free just because its subscription line is small.

A paid tracker earns its place only if it gives you something this setup cannot reliably maintain: a fixed prompt panel, per-engine results, domain-level citations, and a usable change log. Even then, tracking is observation. It does not write the page, fix a crawl problem, or improve a conversion path.

That is why [groas runs paid and organic search as an autonomous growth engine](https://groas.com/for-businesses) rather than selling an advisory dashboard. Its specialized models execute campaign and content work continuously, with action logs and a named human strategist responsible for direction and accountability. If you keep AI visibility in-house, keep the sheet and tie each edit to a prompt. If you pay a platform, **buy the work that closes the gap, not another dial that points at it.**

## Frequently Asked Questions

### Why track AI visibility per prompt instead of using one blended score?

A blended score cannot tell you which buyer question sent a citation to your competitor, while a per-prompt, per-engine log can. Recording each prompt separately for ChatGPT, Gemini, and Google AI Overviews shows exactly where a competitor appeared instead of your brand.

### How do I pick the 25 prompts for an AI search tracking panel?

Pull the last 90 days of support tickets from your CRM or call recordings and your Google Search Console query report, looking for comparison and constraint language like vs, alternative, budget, and how to choose. Use buyers' actual phrasing, put your brand name in no more than five prompts, and freeze the wording for at least 30 days.

### How many times should I run each prompt on each engine?

Three times per engine. Sources and wording can change between runs of the same prompt, so three separate responses give you a frequency to inspect instead of one snapshot standing in for the whole market. For 25 prompts that means 75 records per engine, or 225 observations per cycle.

### What is the difference between a brand mention and a brand citation in AI answers?

A brand named in passing did not necessarily earn a link or a recommendation, so the tracker scores mentions, citations, and recommendations as three different events. Answers where your brand is absent are also retained, because without them your mention percentage has no denominator.

### How often should I run the AI visibility collection cycle?

Once per week on a fixed schedule, with Tuesday 06:00 UTC suggested as a workable setting. Keep the timing fixed so a shift in schedule does not masquerade as a visibility change. If a request fails after three retries with exponential backoff, log the failure rather than manufacturing a zero.

### How do I link site changes to movements in AI search answers?

Create a change_log and, within 24 hours of an edit going live, record its date, URL, type, target prompt IDs, and hypothesis. When you have enough history, compare the targeted prompts for four weeks before and four weeks after the change, engine by engine, including competitor citations.

### How do I verify the tracker is working and find my first fix?

After collection, count 225 rows for a complete cycle, check that every response has a brand_mentioned value, and inspect cited URLs against the raw payload. Then find an unbranded question where you appeared in zero of three runs and a competitor was cited in all three, revise your page for the same clarity without copying the competitor's claims, and watch that prompt over the next four weekly runs.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#webpage","url":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in","name":"Build Your Own Per-Prompt AI Search Tracker for ChatGPT, Gemini, and AI Overviews","description":"Build a 25-prompt tracker that logs mentions, citations, and competitors by AI search engine, then checks each site change against weekly results.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca4a57c7-3d3c-485d-b064-a07899837189.png"},"breadcrumb":{"@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#breadcrumb"},"dateModified":"2026-10-09T05:19:57.602Z","mainEntity":{"@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Build Your Own Per-Prompt AI Search Tracker for ChatGPT, Gemini, and AI Overviews","item":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#article","url":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in","mainEntityOfPage":{"@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Build Your Own Per-Prompt AI Search Tracker for ChatGPT, Gemini, and AI Overviews","description":"Build a 25-prompt tracker that logs mentions, citations, and competitors by AI search engine, then checks each site change against weekly results.","datePublished":"2026-10-09T05:19:57.541Z","dateModified":"2026-10-09T05:19:57.602Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/ca4a57c7-3d3c-485d-b064-a07899837189.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"prompt, brand, search, engine, change, tracker, each, competitor","wordCount":1965,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/build-a-per-prompt-ai-search-tracker-in#faq","mainEntity":[{"@type":"Question","name":"Why track AI visibility per prompt instead of using one blended score?","acceptedAnswer":{"@type":"Answer","text":"A blended score cannot tell you which buyer question sent a citation to your competitor, while a per-prompt, per-engine log can. Recording each prompt separately for ChatGPT, Gemini, and Google AI Overviews shows exactly where a competitor appeared instead of your brand."}},{"@type":"Question","name":"How do I pick the 25 prompts for an AI search tracking panel?","acceptedAnswer":{"@type":"Answer","text":"Pull the last 90 days of support tickets from your CRM or call recordings and your Google Search Console query report, looking for comparison and constraint language like vs, alternative, budget, and how to choose. Use buyers' actual phrasing, put your brand name in no more than five prompts, and freeze the wording for at least 30 days."}},{"@type":"Question","name":"How many times should I run each prompt on each engine?","acceptedAnswer":{"@type":"Answer","text":"Three times per engine. Sources and wording can change between runs of the same prompt, so three separate responses give you a frequency to inspect instead of one snapshot standing in for the whole market. For 25 prompts that means 75 records per engine, or 225 observations per cycle."}},{"@type":"Question","name":"What is the difference between a brand mention and a brand citation in AI answers?","acceptedAnswer":{"@type":"Answer","text":"A brand named in passing did not necessarily earn a link or a recommendation, so the tracker scores mentions, citations, and recommendations as three different events. Answers where your brand is absent are also retained, because without them your mention percentage has no denominator."}},{"@type":"Question","name":"How often should I run the AI visibility collection cycle?","acceptedAnswer":{"@type":"Answer","text":"Once per week on a fixed schedule, with Tuesday 06:00 UTC suggested as a workable setting. Keep the timing fixed so a shift in schedule does not masquerade as a visibility change. If a request fails after three retries with exponential backoff, log the failure rather than manufacturing a zero."}},{"@type":"Question","name":"How do I link site changes to movements in AI search answers?","acceptedAnswer":{"@type":"Answer","text":"Create a change_log and, within 24 hours of an edit going live, record its date, URL, type, target prompt IDs, and hypothesis. When you have enough history, compare the targeted prompts for four weeks before and four weeks after the change, engine by engine, including competitor citations."}},{"@type":"Question","name":"How do I verify the tracker is working and find my first fix?","acceptedAnswer":{"@type":"Answer","text":"After collection, count 225 rows for a complete cycle, check that every response has a brand_mentioned value, and inspect cited URLs against the raw payload. Then find an unbranded question where you appeared in zero of three runs and a competitor was cited in all three, revise your page for the same clarity without copying the competitor's claims, and watch that prompt over the next four weekly runs."}}]}]}
```