---
title: "Your AI Visibility Score Is One Run Dressed Up as a Trend"
description: "A single AI answer cannot tell you how often buyers see your brand. Test repeated runs, prompt variants and controls before you move the budget."
url: "https://groas.com/post/your-ai-visibility-score-is-a-coin-flip"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/bcb3c9e2-ed4c-4aa6-b630-f1ae09382b0a.png"
published: "2026-10-10T05:18:55.991Z"
modified: "2026-10-10T05:18:56.057Z"
---

October 10, 2026 · 10 min read

# Your AI Visibility Score Is One Run Dressed Up as a Trend

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A single die in a tiny navy suit and red tie stands on a boardroom table like an executive, mocking chance results presented as authoritative metrics.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/bcb3c9e2-ed4c-4aa6-b630-f1ae09382b0a.png)

In this article

1. [Your AI visibility score is not a score](#your-ai-visibility-score-is-not-a-score)
2. [Why the usual visibility score lies](#why-the-usual-visibility-score-lies)
3. [The test I would run before moving a dollar](#the-test-i-would-run-before-moving-a-dollar)
4. [Find the buyer constraints that bring you into the answer](#find-the-buyer-constraints-that-bring-you-into-the-answer)
5. [The in-house version: one spreadsheet, one afternoon](#the-in-house-version-one-spreadsheet-one-afternoon)
6. [Pay for action, not another alarm](#pay-for-action-not-another-alarm)
7. [Hand back the single-run report](#hand-back-the-single-run-report)

## Your AI visibility score is not a score

**One prompt, one run, one screenshot is not AI visibility measurement.** It is a sample dressed up for a budget meeting.

Picture the slide: your brand has an “AI Visibility Score” of 43% across ChatGPT, Gemini and Google AI Overviews. A vendor assembled 30 prompts, ran each one through an API once, found your name in 13 answers and divided by 30. A rival sits at 65%. The executive team approves another $4,000 a month to close the gap.

That budget is chasing a number the vendor has not earned. If a media buyer evaluated an ad account from one impression on a Tuesday morning, I would revoke their access before lunch. Yet AI visibility trackers routinely treat one batch of answers like a 2015 rank report. They can tell you what appeared in that batch. They cannot tell you that buyers see your brand 43% of the time.

Run the prompts again and the answers may change. In empirical testing across category questions, researchers found that [within-prompt resampling alone accounted for 34.8% of output variance](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGiBc3E2MOAH0IP6KO00XMNjGm28w7SrD51eMbFaKanK5OFYxMbuJN_WoLngdAHEmIEZhcyuba9j1I5oOUChQImX0uVV8GzRFBO9dstaJM6JBxNQdD2CMDaIw=), even at a low decoding temperature. A change in your score does not, by itself, mean a competitor published better content or your agency’s schema edits worked. It may be sampling noise with a logo on top.

## Why the usual visibility score lies

### The answer changes even when the prompt does not

Traditional rank tracking starts with a relatively stable object: a results page assembled from an index. Search results can change, but a tracker can return to the same query and compare positions. An LLM answer is not a position on a shelf. The model generates text token by token, drawing from probabilities; interfaces that retrieve live material add another moving part.

That distinction matters most when the question asks for judgment rather than a simple fact. In tracking of brand outputs under identical prompts, [only 37% of featured brands remained consistent across repeated runs](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQEubWZxwAMJKNH1l_FEXf1sSwycy-wIpjnGCNawcpB5qVG3WqXfjYtXaLXebMfUwsjINoKMBl9WS403n2LoHBUofU2Z7o4UQhFbfndpPAnBKUZZLhkyI9lyhhAlMid6s7n47cwDWEpn_sOZrsgmubxoHalhlvKrnw=). Simple factual queries diverged by 5% to 10%; complex recommendation and evaluation queries showed **40% to 60% output divergence**. A weekly slide that drops from 55% to 30% cannot tell you whether your standing degraded unless the tracker first establishes how much the answer normally moves.

**Repeat the identical prompt before calling a change a trend.**

### Buyers do not all ask the same sentence

Suppose a buyer wants warehouse software. One asks for the “best warehouse management software for mid-market retail.” Another wants “top-rated WMS tools for 50-person logistics teams.” A third asks what to buy after outgrowing Fishbowl. A tracker that tests only the first sentence does not measure the whole buying conversation.

[Prompt-sensitivity audits](https://suprmind.ai/hub/methodology/prompt-sensitivity/) put the error margin from evaluating a single query variant at ±40%. Swapping “best” for “top” changed featured vendors by 15% to 30%; intent modifiers changed the list by 40% to 60%. Those are not cosmetic edits. They alter what the buyer is asking the model to solve.

**Test variants of the buyer’s problem, not your favorite wording of it.**

### A blended engine score hides the useful result

Commercial trackers often roll ChatGPT, Gemini, Perplexity and Google AI Overviews into one composite. That makes a clean chart, but it buries which interface actually produced the mention. The systems have different models and retrieval behavior. In an analysis of [300 business software queries across AI interfaces](https://www.reddit.com/r/SEO_for_AI/comments/1nkari2/how_much_does_ai_search_responses_vary_for_the/), broad queries showed 70% recommendation variance across platforms. Google AI Overviews also showed greater run-to-run volatility than the standalone chat models in that analysis.

If one engine recommends you and another does not, an average will conceal the difference you need to investigate. **Keep results separate by engine.** A composite can wait until the underlying answers make sense.

## The test I would run before moving a dollar

When I managed search accounts by hand, I did not kill an ad group after three impressions. I waited for volume, checked what else had changed and decided what result would justify an action. AI visibility needs the same operator discipline. Here is the question: **does your brand appear consistently in unprompted commercial answers, or did one batch happen to include it?**

This is a prospective test, not a claim that I ran your prompts. Set it up before you open the dashboard.

### Set up a panel that does not give away the answer

Remove your brand name from every measured query. A prompt such as “How does \[Brand Name] compare to Competitor X?” invites the model to repeat your name, then rewards the tracker for spotting it. In one [documented audit](https://insightwonder.com/features/ai-visibility/), a client’s reported 13% visibility came entirely from a prompt that named the company; its unprompted visibility across commercial category searches was zero.

Build a panel of **20 to 30 unbranded buyer prompts** from sales conversations, or use a structured starting point such as our [60 AI search prompts sorted by buyer intent](https://groas.com/post/the-prompt-list-i-d-track-if-i-owned-you). Include problem-aware questions, category comparisons and high-intent questions with specific constraints. Write down which group each prompt belongs to. Do not quietly replace the panel next month because the first version produced an ugly score.

### Hold the setup steady, then repeat the runs

For each engine you choose to track, run every prompt **3 to 5 times** within a tight testing window. Log the engine, exact phrasing, run number and full answer. Keep the setup consistent between runs and testing dates so a change in your own procedure does not masquerade as a change in visibility.

Twenty iterations per prompt can produce cleaner confidence intervals, as our [AI visibility test protocol](https://groas.com/post/run-every-prompt-20-times-a-test-protoco) explains. But a [variance decomposition](https://vertexaisearch.cloud.google.com/grounding-api-redirect/AUZIYQGiBc3E2MOAH0IP6KO00XMNjGm28w7SrD51eMbFaKanK5OFYxMbuJN_WoLngdAHEmIEZhcyuba9j1I5oOUChQImX0uVV8GzRFBO9dstaJM6JBxNQdD2CMDaIw=) found sharply diminishing returns beyond five repeats per prompt-engine cell. For a lean audit, I would spend the next unit of effort on another buyer phrasing or engine, not another identical run.

![An open laboratory notebook contrasts one coin toss with a tray of rolled dice.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/1ba2e087-2a84-464a-8ae5-d1a785bc474b.png)

### Add controls that should stay dark

A PPC operator uses negative keywords to catch irrelevant traffic and checks that test transactions do not count as real conversions. Give the prompt panel the same sanity check: add **five to ten control queries where your brand should not appear**.

If you sell B2B accounting software for medical clinics, controls might ask about oil-and-gas logistics, residential landscaping invoicing or freelance design templates. A brand mention there is not evidence of astonishing reach. It is a reason to inspect the answer and your testing pipeline. If controls start lighting up, do not celebrate the visibility score. Fix the test.

### Measure what the model actually did

“Visible” is too blunt a label. For every answer, record whether your brand was:

- **Recommended** as a vendor solution.
- **Mentioned** without a recommendation.
- **Cited** through a linked source.
- **Absent.**

Keep the citation field separate; an answer can cite your content while recommending someone else. In an audit of 1,796 commercial category queries, [Asva AI](https://asvaai.com/features/brand-visibility-tracker) reported that its domain was cited 142 times but recommended as a vendor zero times. [Crunchjunkie](https://crunchjunkie.io/blog/ai-tracking) likewise found cases where ChatGPT cited a company’s technical post to explain a problem, then recommended a competitor. **A source citation is not a buying recommendation.** Count both, but do not sell one to leadership as the other.

![A split diagram contrasts an informational citation with a vendor recommendation.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4ce6f521-88cf-4eff-aa72-39420f5d5d50.png)

### Decide what would change your mind

Before comparing months, write down the threshold that would make you investigate or act. My practical rule for this panel: treat a shift of less than **15 percentage points**, or one that does not persist across two monthly testing windows, as insufficient grounds for a budget move. That is an operating guardrail, not a universal confidence interval. A different panel may need a different threshold; the important part is choosing it before you see the result.

I expect broad recommendation prompts to move around more than tightly constrained ones because a vague request leaves more plausible vendors in play. I also expect repeats to expose some apparent wins as ordinary variation. If the recommendation frequency rises beyond your threshold, persists on the next check and does not appear in the controls, investigate what changed and whether it reaches pipeline. If it does not, leave the budget alone. Our [guide to measuring AI search visibility without trusting one screenshot](https://groas.com/post/how-to-measure-your-brand-s-visibility-i-2) goes deeper on that decision.

## Find the buyer constraints that bring you into the answer

Broad prompts such as “best project management software” are the AI-answer equivalent of broad-match head terms. They attract well-funded incumbents and give the model little reason to pick a specialist. In the [300-prompt analysis](https://www.reddit.com/r/SEO_for_AI/comments/1nkari2/how_much_does_ai_search_responses_vary_for_the/), broad queries showed 70% recommendation variance; heavily constrained queries reduced it to 25%.

Mine closed-lost notes and sales transcripts for the friction buyers mention near a decision: headcount limits, compliance requirements, legacy software migrations and integrations. Turn those into unbranded prompt variants. You are not trying to coax the model into saying your name. You are testing whether it recommends you when a buyer describes the problem you actually solve.

That distinction also tells you what to do with a weak result. If your brand never appears for a broad category term but appears repeatedly for a relevant constraint, do not flatten those observations into one “share of voice” score. The constrained question shows where your case is credible. **Report visibility by buyer problem, not just by category.**

## The in-house version: one spreadsheet, one afternoon

You do not need a $500-a-month subscription just to learn whether the same prompts produce the same answers. A [single spreadsheet and a two-hour monthly audit](https://www.amicited.com/blog/diy-ai-visibility-tracking/) can give a small team a usable starting point. Track seven fields:

- Date
- Exact prompt phrasing
- Buyer intent stage
- Engine
- Run number
- Recommendation status: Recommended, Mentioned or Absent
- Footnote URL, if present

Keep the full answers as well. Without them, you cannot check whether your labels captured what the model said. On the first Tuesday of each month, run the same 25 core prompts through each chosen engine five times. For each prompt and engine, calculate **Recommendation Frequency**: runs that recommended you divided by total runs. Calculate **Citation Rate** separately: runs with a linked source divided by total runs.

Basic OpenAI or Google Gemini API scripts can automate execution if you want them to. Twenty-five prompts at five runs require roughly 125 calls **per engine**. Automation saves copying; it does not decide whether a mention is commercially useful or whether a change clears your threshold. Preserve the raw text and the historical log. A colored area chart is not an audit trail.

![A marketer audits AI search answers in a spreadsheet beside an open browser window.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/7611a77a-da70-467f-b905-b08a94079d9d.png)

## Pay for action, not another alarm

Marketers buy software for problems that require operational discipline. Agencies charge retainers to watch dashboards change color. Paying hundreds a month for a tool that pings an LLM, plots the answers and sends alerts to an unread Slack channel is an expensive alarm clock.

An alert about a missing citation does not make a pricing table easier to find, clear a blocked crawler or restructure paid ad groups around unbranded demand. Those are execution problems. **Monitoring earns its fee only when someone can act on a credible signal.**

That is the operating philosophy behind [groas](https://groas.com/): paid search, SEO and AI discoverability belong in one continuous execution system. Specialized models adjust bids, refine landing page assets and expand query coverage around the clock, while a named human strategist owns direction and commercial guardrails. The point is not to give the team another probabilistic chart to decipher. It is to connect what the team learns to work that can affect revenue.

## Hand back the single-run report

The next time a vendor puts an AI visibility score in front of you, ask how many runs produced it. If the answer is one, hand it back. If your brand appears in 7 of 20 answers, you observed **35% in that sample**. You have not established that buyers see you 35% of the time, or that a later 30% means you lost ground.

Report results by prompt group and engine. Show repeated-run frequencies, keep citations distinct from recommendations, and watch the trend alongside downstream pipeline. Then move money only when a change clears the threshold you set before looking.

Ignore that discipline and you will spend your marketing budget optimizing for whichever answer the model happened to give on Tuesday.

## Frequently Asked Questions

### Why is one run of AI visibility prompts not enough to trust the score?

One run is a single sample, not a trend. Even resampling the same prompt changes the output; within-prompt resampling alone accounted for 34.8% of output variance in empirical testing, even at low decoding temperature. A score built from one run per prompt cannot tell you how often buyers actually see your brand.

### Do AI models give the same answer if you ask the exact same prompt twice?

Often they do not. In tracking under identical prompts, only 37% of featured brands remained consistent across repeated runs, and complex recommendation queries showed 40% to 60% output divergence. A weekly score change may just be sampling noise rather than a real shift in your standing.

### How much does changing the wording of a prompt change AI answers?

It can change them a lot. Audits put the error margin from evaluating a single query variant at plus or minus 40%, and swapping 'best' for 'top' changed the featured vendors by 15% to 30%. Intent modifiers changed the recommended list by 40% to 60%, so you should test variants of the buyer's problem rather than one favorite phrasing.

### Should I combine ChatGPT, Gemini and Perplexity results into one AI visibility score?

No, keep the results separate by engine at first. In an analysis of 300 business software queries, broad queries showed 70% recommendation variance across platforms, and Google AI Overviews were more volatile run to run than standalone chat models. A composite average can hide which interface actually produces or withholds your mentions.

### What kind of prompts should I use to measure AI visibility?

Build a panel of 20 to 30 unbranded buyer prompts drawn from sales conversations, covering problem-aware, category-comparison and high-intent questions. A prompt that contains your brand name invites the model to repeat it; in one documented audit, a reported 13% visibility came entirely from brand-named prompts while unprompted visibility was zero.

### How many times should I repeat each prompt when testing AI answers?

Run every prompt 3 to 5 times per engine within a tight testing window, logging the engine, exact phrasing, run number and full answer. Variance analysis found sharply diminishing returns beyond five repeats, so extra effort is better spent on another buyer phrasing or engine than on more identical runs.

### How do control queries work in an AI visibility test?

Add five to ten control queries on unrelated topics where your brand should not appear, the way a PPC operator uses negative keywords. If your brand shows up in those controls, the result indicates a problem with the answer or the testing pipeline, not impressive reach, and you should fix the test before trusting any score.

### Does a citation in an AI answer count as a recommendation?

No. An answer can cite your content as a source while recommending a competitor as the vendor, and audits found cases where a domain was cited 142 times but recommended as a vendor zero times. Count citations and recommendations separately and never report one to leadership as the other.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#webpage","url":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip","name":"Your AI Visibility Score Is One Run Dressed Up as a Trend","description":"A single AI answer cannot tell you how often buyers see your brand. Test repeated runs, prompt variants and controls before you move the budget.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/bcb3c9e2-ed4c-4aa6-b630-f1ae09382b0a.png"},"breadcrumb":{"@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#breadcrumb"},"dateModified":"2026-10-10T05:18:56.057Z","mainEntity":{"@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Your AI Visibility Score Is One Run Dressed Up as a Trend","item":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#article","url":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip","mainEntityOfPage":{"@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Your AI Visibility Score Is One Run Dressed Up as a Trend","description":"A single AI answer cannot tell you how often buyers see your brand. Test repeated runs, prompt variants and controls before you move the budget.","datePublished":"2026-10-10T05:18:55.991Z","dateModified":"2026-10-10T05:18:56.057Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/bcb3c9e2-ed4c-4aa6-b630-f1ae09382b0a.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"visibility, score, run, prompt, visibility score, brand, answer, prompts","wordCount":2020,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/your-ai-visibility-score-is-a-coin-flip#faq","mainEntity":[{"@type":"Question","name":"Why is one run of AI visibility prompts not enough to trust the score?","acceptedAnswer":{"@type":"Answer","text":"One run is a single sample, not a trend. Even resampling the same prompt changes the output; within-prompt resampling alone accounted for 34.8% of output variance in empirical testing, even at low decoding temperature. A score built from one run per prompt cannot tell you how often buyers actually see your brand."}},{"@type":"Question","name":"Do AI models give the same answer if you ask the exact same prompt twice?","acceptedAnswer":{"@type":"Answer","text":"Often they do not. In tracking under identical prompts, only 37% of featured brands remained consistent across repeated runs, and complex recommendation queries showed 40% to 60% output divergence. A weekly score change may just be sampling noise rather than a real shift in your standing."}},{"@type":"Question","name":"How much does changing the wording of a prompt change AI answers?","acceptedAnswer":{"@type":"Answer","text":"It can change them a lot. Audits put the error margin from evaluating a single query variant at plus or minus 40%, and swapping 'best' for 'top' changed the featured vendors by 15% to 30%. Intent modifiers changed the recommended list by 40% to 60%, so you should test variants of the buyer's problem rather than one favorite phrasing."}},{"@type":"Question","name":"Should I combine ChatGPT, Gemini and Perplexity results into one AI visibility score?","acceptedAnswer":{"@type":"Answer","text":"No, keep the results separate by engine at first. In an analysis of 300 business software queries, broad queries showed 70% recommendation variance across platforms, and Google AI Overviews were more volatile run to run than standalone chat models. A composite average can hide which interface actually produces or withholds your mentions."}},{"@type":"Question","name":"What kind of prompts should I use to measure AI visibility?","acceptedAnswer":{"@type":"Answer","text":"Build a panel of 20 to 30 unbranded buyer prompts drawn from sales conversations, covering problem-aware, category-comparison and high-intent questions. A prompt that contains your brand name invites the model to repeat it; in one documented audit, a reported 13% visibility came entirely from brand-named prompts while unprompted visibility was zero."}},{"@type":"Question","name":"How many times should I repeat each prompt when testing AI answers?","acceptedAnswer":{"@type":"Answer","text":"Run every prompt 3 to 5 times per engine within a tight testing window, logging the engine, exact phrasing, run number and full answer. Variance analysis found sharply diminishing returns beyond five repeats, so extra effort is better spent on another buyer phrasing or engine than on more identical runs."}},{"@type":"Question","name":"How do control queries work in an AI visibility test?","acceptedAnswer":{"@type":"Answer","text":"Add five to ten control queries on unrelated topics where your brand should not appear, the way a PPC operator uses negative keywords. If your brand shows up in those controls, the result indicates a problem with the answer or the testing pipeline, not impressive reach, and you should fix the test before trusting any score."}},{"@type":"Question","name":"Does a citation in an AI answer count as a recommendation?","acceptedAnswer":{"@type":"Answer","text":"No. An answer can cite your content as a source while recommending a competitor as the vendor, and audits found cases where a domain was cited 142 times but recommended as a vendor zero times. Count citations and recommendations separately and never report one to leadership as the other."}}]}]}
```