---
title: "The 30-Day AEO Holdout: Test Citations Before You Sign"
description: "A 30-day AEO tool test for agencies: split buyer prompts into worked and untouched sets across three client domains, then measure citation lift and team hours."
url: "https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/feaed93d-665a-4719-874e-8430a4434055.png"
published: "2026-10-12T05:16:57.014Z"
modified: "2026-10-12T05:16:57.080Z"
---

October 12, 2026 · 9 min read

# The 30-Day AEO Holdout: Test Citations Before You Sign

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A skeptical man crouches in a greenhouse, measuring a 'WORKED' seedling against an identical 'CONTROL' one beside an unsigned contract.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/feaed93d-665a-4719-874e-8430a4434055.png)

In this article

1. [Setup: three domains with different starting points](#setup-three-domains-with-different-starting-points)
2. [During the trial: record what the tool actually ships](#during-the-trial-record-what-the-tool-actually-ships)
3. [Every Monday: measure the change in the gap](#every-monday-measure-the-change-in-the-gap)
4. [Before renewal: do the ten-client math](#before-renewal-do-the-ten-client-math)
5. [Day 30: decide what the result changes](#day-30-decide-what-the-result-changes)

An AEO dashboard can show a rising visibility score without earning your client a single new citation. I have seen the ad-copy version of this trick: CTR moves, sales do not, and the chart gets called momentum. AI search gives it a new label. The dashboard reports citations. It does not create them.

Before you put an AEO tool on five or ten client retainers, run a holdout. Pick three client domains. Split each client’s buyer prompts into a worked set you act on and a control set you leave alone. After 30 days, ask whether citations improved _more_ for the worked prompts than for the controls, and count the team hours it took. Work shipped, fixes applied, citations earned. **If the worked set does not pull away from the control set, the tool has not proved its worth as an execution service.**

## Setup: three domains with different starting points

Pick one strong domain that already gets cited sometimes, one average domain that ranks in Google but rarely appears in AI answers, and one domain with almost no presence. I used to test ad copy against different starting points for the same reason. Test only a winner and you cannot tell whether the tool helped. Test only a loser and you can blame the account. Three starting points ask the tool to hold a lead, move the middle, and build from nearly zero.

Keep other activity as still as you can for 30 days. No other content vendor should touch these domains; no site migration or review blitz should run through the test. Tell each client what you are doing and why some prompts will stay untouched. **The control set is what stops you from crediting the tool for an engine reshuffle.** If you cannot keep major changes out of the test, choose different domains or postpone it.

### Write buyer prompts before the vendor sees the list

Do not let the tool choose its own exam questions. For each client, write 30 to 50 prompts in the words buyers use when they are close to a decision: _best_, _cost_, _compare_, _near me_, _who should I hire_, _what should I buy_. For a garage-door client, that might mean _best emergency garage door repair in Dallas_, _how much does a garage door spring replacement cost_, and _Chamberlain vs Amarr reliability_.

Ten brand prompts do not count. You are not testing whether ChatGPT can spell the client’s name. You are testing whether an answer points to the client when money is on the table.

If your team is small, cap the list at 40 per client. That gives you 120 prompts across three domains: enough to look for a pattern, small enough that someone will run the checks every Monday without faking them. Record every prompt word for word. Change the wording mid-test and that prompt needs a new baseline. I learned that lesson when someone tweaked an ad headline on day 12 and wanted to call it the same test. It was not.

### Divide each list into worked and untouched prompts

Before the split, group obviously similar prompts so one half does not get all the easy brand-adjacent questions or all the near-duplicates. Then randomly divide each client’s list as evenly as possible. Half become **worked prompts**: the tool and your team can act on those topics. The other half become **control prompts**: no new page, schema tweak, or rewrite should target them for 30 days.

Keep a record of which prompt belongs to which set, and do not swap prompts later because a result looks inconvenient. No holdout is a perfect wall: a sitewide change may affect both halves. That is why you log what changed rather than pretending the control exists in another universe. But an untouched set in the same account, month, and engines is still a better check than a before-and-after screenshot. [Identical prompts can return different brands and URLs on repeat runs](https://thepromptinsider.com/aeo/what-is-ai-search-volatility/). Without a control, engine churn can look an awful lot like earned lift.

### Take the day-0 baseline under repeatable conditions

On day 0, run the full prompt list through ChatGPT, Perplexity, Gemini, and Google AI Overviews. For each run, record whether the client was cited and which URL the answer cited. Use clean sessions and the same location and language each time. Test each engine separately.

For high-priority buyer prompts, run the check on two days and keep the appearance frequency, not just a single yes-or-no snapshot. [Appearance rate across repeats helps separate signal from noise](https://thepromptinsider.com/aeo/what-is-ai-search-volatility/). Use one sheet for the [fixed buyer-prompt panel](https://groas.com/post/how-to-measure-your-brand-s-visibility-i-2): client, prompt, set, engine, date, cited yes/no, cited URL, and brand mentioned without a link. Keep the raw answers available for manual checks. That sheet is your control chart for the next four Mondays.

## During the trial: record what the tool actually ships

A monitoring subscription tracks mentions, share of voice, and citations. Your team still writes every page and ships every fix. That is the distinction [HubSpot’s 2026 AEO cost guide draws between $29-to-$489-per-month trackers and $9,000-to-$15,000-per-month programs that bundle content, schema, and off-site work](https://blog.hubspot.com/marketing/how-much-does-aeo-cost). If a vendor sells you monitoring, grade it as monitoring. If it sells you execution, make it prove execution on the worked prompts.

Log these each week, by domain:

- **Pages shipped to the CMS** for worked topics without your writers rewriting them.
- **Technical and structured-data fixes closed** without a developer ticket.
- **Prompt checks completed**, including any caps or extra charges.
- **Agency hours spent** checking, editing, publishing, chasing tickets, and cleaning up reports.

If the tool flags missing schema in week one and it is still flagged in week four, do not count the alert as a fix. What the deck calls _insights_, I call homework. By day 14, if nothing has shipped to the worked prompts and the tool has not closed at least one technical gap per domain, stop grading it as an execution service. It may still be useful monitoring. That is a different thing to resell.

Demand raw proof while the work happens. For every citation the dashboard claims, ask for the query, the engine, and the source URL the answer cited. No rolled-up visibility score in place of those fields. [HubSpot warns against vendor-defined scores](https://blog.hubspot.com/marketing/how-much-does-aeo-cost): answers change, and a score can rise without a new client URL appearing. Check [whether the page earned the citation](https://groas.com/post/the-prompt-trigger-test-find-out-which-b) yourself. Open the answer, find the link, confirm the client page, and log any mismatch. A screenshot is not a substitute for that row in the sheet.

## Every Monday: measure the change in the gap

Rerun the full list under the day-0 conditions: same wording, location, language, and clean sessions, with each engine checked separately. For each set and engine, calculate the share of prompts that earned a citation. Then compare the worked rate with the control rate.

**The result is not just the day-30 gap. It is how much that gap changed since day 0.** If worked prompts finish at 35% cited and controls finish at 20%, they are 15 percentage points apart. That is not automatically 15 points of lift. If they were already 15 points apart before anyone shipped a page, the test has shown no new separation. Subtract the day-0 gap from the day-30 gap before you call anything earned.

Read the weekly line as well as the final number. Cited pages turn over fast, with [median cited-page half-life around 4.5 weeks and ChatGPT churning faster than Perplexity](https://thepromptinsider.com/aeo/what-is-ai-search-volatility/). One good Monday is weather. Several Mondays with a growing worked-over-control gap give you a better reason to take the result seriously. Keep the engine-level results visible; a blended figure should not hide where the change happened.

![Lab notebook dividing buyer prompts into worked and untouched control sets](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/1e7b37c3-f238-4e36-9134-11730e54d712.png)

Log hours as carefully as citations, because hours are the margin. I used to tell clients tracking was half the battle. I was wrong. Tracking without shipping is a subscription to bad news. If the worked set needs six hours per domain per week from your team, multiply that by ten clients before you sign anything. A tool can improve a citation chart and still leave you with a service you cannot profitably deliver.

## Before renewal: do the ten-client math

Vendors price by domain, prompt bundle, or engine. Your P\&L prices by client. A hands-on test of 15 trackers lists [Semrush AI Visibility at $99 per month for one domain and 25 daily prompts, plus $99 per extra domain and $60 per extra 50 prompts](https://www.position.digital/blog/best-ai-visibility-tracking-tools/). At agency scale, a [vendor comparison puts 10 clients at 40 prompts each at $399 per month on Geoptie Enterprise, $489 on Otterly Premium, $795 on Peec Scale, $990 on Semrush with a 25-prompt-per-client cap, and $4,089 on Profound Agency Growth](https://geoptie.com/blog/best-geo-tools-for-agencies). Those are plan figures, not your delivered cost.

Add the hours. A $399 plan that takes five hours per client per week costs you more labor than a $795 plan that takes one. For a ten-client forecast, divide the monthly tool fee by ten, then add each client’s weekly agency hours multiplied by your blended hourly rate and the weeks you expect to work that month. **If the fully loaded cost per domain does not fit inside the retainer with margin left over, the trial fails the reseller test**, whatever the dashboard says.

## Day 30: decide what the result changes

A persuasive win has a mechanism you can point to. Pages or fixes went live for worked topics; the worked citation rate then pulled ahead of the controls relative to baseline; the cited URLs match the work; and the hours fit the retainer. Look for separation on more than one engine rather than staking a renewal on a single screenshot. One hands-on review of trackers [cites SparkToro finding less than a 1% chance that ChatGPT and Google AI repeat the same brand list](https://www.position.digital/blog/best-ai-visibility-tracking-tools/). My ad-testing rule still applies: a 10-point gap that persists for three straight Mondays beats a 25-point spike that vanishes by Friday.

![Two diverging trend lines for worked and control prompts pinned to a board](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/99b44e27-b727-48de-bc3b-68b0141f915d.png)

If both sets move together, up or down, do not hand the tool credit for the shared movement. If the worked line rises only inside the vendor dashboard while manual checks keep finding the same old competitors, log a reporting miss. Reviewers have flagged [only moderate consistency between platform-reported citations and manual checks](https://www.vismore.ai/blog/best-aeo-tools-2026). A miss belongs in the result, not in the footnotes.

This protocol will not fit every client. Skip domains mid-migration or those with fewer than 20 buyer prompts worth tracking; a thin prompt set makes each weekly read too jumpy to carry the decision. Otherwise, complete the scorecard before you consider an annual contract. If a vendor will not let you run the holdout, that is also an answer.

- **Worked prompts cited:** \_\_% on day 0 to \_\_% on day 30, by engine
- **Control prompts cited:** \_\_% on day 0 to \_\_% on day 30, by engine
- **Gap earned:** day-30 worked-minus-control gap, minus the day-0 gap: \_\_ percentage points
- **Weekly pattern:** gap held or grew for \_\_ consecutive Mondays
- **Pages shipped to the CMS** without your writers rewriting them: \_\_
- **Technical and schema fixes closed** without a developer ticket: \_\_
- **Tracking caps hit:** none, or list the limit and charge
- **Agency hours per domain per week:** \_\_; projected hours across ten domains: \_\_
- **Fully loaded monthly cost per domain** at ten domains: $\_\_
- **Manual-check match rate:** dashboard citations confirmed in \_\_ of \_\_ checks

If the gap grows and holds, the cited URLs check out, and the hours fit your retainer, you have something worth testing across more client domains. Keep the control set running for another 30 days; engines drift, and a renewal should not rest on one month’s luck. If the gap stays flat after supposed execution, do not put a reporting fee on five retainers and call it a service. That is the bar I hold any [done-for-you AEO service](https://groas.com/for-businesses) to: shipped pages, closed fixes, and citations your sheet can prove. Everything else is a prompt meter with better fonts.

## Frequently Asked Questions

### How can an agency test whether an AEO tool actually earns client citations before signing?

Run a 30-day holdout across three client domains. Split each client's buyer prompts into a worked set the tool and team act on and a control set left untouched, then compare citation lift between the sets and count the team hours it took. If the worked set does not pull away from the controls, the tool has not proved itself as an execution service.

### Why should I test an AEO tool on three domains with different starting points?

Testing only a strong domain means you cannot tell whether the tool helped, and testing only a weak domain lets you blame the account instead. Three starting points ask the tool to hold a lead, move the middle domain, and build one from nearly zero.

### Who should write the test prompts for an AEO tool trial?

You should write 30 to 50 prompts per client before the vendor sees the list, rather than letting the tool choose its own exam questions. Use the words buyers use near a decision, such as best, cost, compare, and who should I hire. Keep the wording identical all test; changing a prompt mid-test means it needs a new baseline.

### How do I split buyer prompts into worked and control sets?

First group obviously similar prompts so one set does not get all the easy near-duplicates, then randomly divide each client's list as evenly as possible. Half become worked prompts the tool and team can act on, and half become control prompts that get no new pages, schema tweaks, or rewrites for 30 days. Keep a record and do not swap prompts later.

### How do I take a day-0 baseline for an AEO holdout test?

On day 0, run the full prompt list through ChatGPT, Perplexity, Gemini, and Google AI Overviews, testing each engine separately with clean sessions and the same location and language each time. Record whether the client was cited and which URL was cited, and run high-priority prompts on two days to keep an appearance frequency rather than a single snapshot.

### How do I calculate citation lift from an AEO tool trial?

Calculate the share of prompts earning a citation for each set and engine, then subtract the day-0 gap from the day-30 gap. A 15-percentage-point day-30 gap is not 15 points of lift if the sets were already 15 points apart before any work shipped. Also watch the weekly line, since a gap that grows over several Mondays is more reliable than one spike.

### When should I stop grading an AEO tool as an execution service during the trial?

By day 14, if nothing has shipped to the worked prompts and the tool has not closed at least one technical gap per domain, stop grading it as an execution service. An alert that stays flagged from week one to week four is homework, not a fix. The tool may still be useful monitoring, but that is a different thing to resell.

### How should I decide whether an AEO tool fits my agency's retainers before renewal?

Divide the monthly tool fee by ten clients, then add each client's weekly agency hours multiplied by your blended hourly rate and the weeks worked that month. A $399 plan taking five hours per client per week costs more labor than a $795 plan taking one. If the fully loaded cost per domain does not fit inside the retainer with margin left over, the trial fails the reseller test.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#webpage","url":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any","name":"The 30-Day AEO Holdout: Test Citations Before You Sign","description":"A 30-day AEO tool test for agencies: split buyer prompts into worked and untouched sets across three client domains, then measure citation lift and team hours.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/feaed93d-665a-4719-874e-8430a4434055.png"},"breadcrumb":{"@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#breadcrumb"},"dateModified":"2026-10-12T05:16:57.080Z","mainEntity":{"@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"The 30-Day AEO Holdout: Test Citations Before You Sign","item":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#article","url":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any","mainEntityOfPage":{"@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"The 30-Day AEO Holdout: Test Citations Before You Sign","description":"A 30-day AEO tool test for agencies: split buyer prompts into worked and untouched sets across three client domains, then measure citation lift and team hours.","datePublished":"2026-10-12T05:16:57.014Z","dateModified":"2026-10-12T05:16:57.080Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/feaed93d-665a-4719-874e-8430a4434055.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"prompts, test, worked, client, tool, before, each, aeo","wordCount":2004,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/the-30-day-aeo-tool-bake-off-a-test-any#faq","mainEntity":[{"@type":"Question","name":"How can an agency test whether an AEO tool actually earns client citations before signing?","acceptedAnswer":{"@type":"Answer","text":"Run a 30-day holdout across three client domains. Split each client's buyer prompts into a worked set the tool and team act on and a control set left untouched, then compare citation lift between the sets and count the team hours it took. If the worked set does not pull away from the controls, the tool has not proved itself as an execution service."}},{"@type":"Question","name":"Why should I test an AEO tool on three domains with different starting points?","acceptedAnswer":{"@type":"Answer","text":"Testing only a strong domain means you cannot tell whether the tool helped, and testing only a weak domain lets you blame the account instead. Three starting points ask the tool to hold a lead, move the middle domain, and build one from nearly zero."}},{"@type":"Question","name":"Who should write the test prompts for an AEO tool trial?","acceptedAnswer":{"@type":"Answer","text":"You should write 30 to 50 prompts per client before the vendor sees the list, rather than letting the tool choose its own exam questions. Use the words buyers use near a decision, such as best, cost, compare, and who should I hire. Keep the wording identical all test; changing a prompt mid-test means it needs a new baseline."}},{"@type":"Question","name":"How do I split buyer prompts into worked and control sets?","acceptedAnswer":{"@type":"Answer","text":"First group obviously similar prompts so one set does not get all the easy near-duplicates, then randomly divide each client's list as evenly as possible. Half become worked prompts the tool and team can act on, and half become control prompts that get no new pages, schema tweaks, or rewrites for 30 days. Keep a record and do not swap prompts later."}},{"@type":"Question","name":"How do I take a day-0 baseline for an AEO holdout test?","acceptedAnswer":{"@type":"Answer","text":"On day 0, run the full prompt list through ChatGPT, Perplexity, Gemini, and Google AI Overviews, testing each engine separately with clean sessions and the same location and language each time. Record whether the client was cited and which URL was cited, and run high-priority prompts on two days to keep an appearance frequency rather than a single snapshot."}},{"@type":"Question","name":"How do I calculate citation lift from an AEO tool trial?","acceptedAnswer":{"@type":"Answer","text":"Calculate the share of prompts earning a citation for each set and engine, then subtract the day-0 gap from the day-30 gap. A 15-percentage-point day-30 gap is not 15 points of lift if the sets were already 15 points apart before any work shipped. Also watch the weekly line, since a gap that grows over several Mondays is more reliable than one spike."}},{"@type":"Question","name":"When should I stop grading an AEO tool as an execution service during the trial?","acceptedAnswer":{"@type":"Answer","text":"By day 14, if nothing has shipped to the worked prompts and the tool has not closed at least one technical gap per domain, stop grading it as an execution service. An alert that stays flagged from week one to week four is homework, not a fix. The tool may still be useful monitoring, but that is a different thing to resell."}},{"@type":"Question","name":"How should I decide whether an AEO tool fits my agency's retainers before renewal?","acceptedAnswer":{"@type":"Answer","text":"Divide the monthly tool fee by ten clients, then add each client's weekly agency hours multiplied by your blended hourly rate and the weeks worked that month. A $399 plan taking five hours per client per week costs more labor than a $795 plan taking one. If the fully loaded cost per domain does not fit inside the retainer with margin left over, the trial fails the reseller test."}}]}]}
```