Is $8K/Month for 'AI SEO' Worth It? Without Measurable Data, You're Paying for Nothing

A Client's Hard Questions About 'AI Visibility'

A marketing lead at a company posted a pointed question on Reddit: their organization had added an "AI Visibility" line item to their existing SEO contract this quarter — same agency, same people — but the monthly fee jumped by $3,000, bringing the total to $8,000.

The real question: what exactly did that extra money buy? The answer was disappointing — a slide deck stuffed with ChatGPT response screenshots, no test prompt set, no before/after comparison, and not a single date stamp. When he asked the agency to provide the actual prompts used for testing, they couldn't deliver.

The post resonated widely because it exposed one of the messiest aspects of the emerging "AI SEO" or Generative Engine Optimization (GEO) space: the concept is red-hot, but delivery standards are extremely vague.

Screenshots Are Not a Methodology

The poster's core challenge was actually quite sophisticated — he wasn't questioning whether "AI visibility" has value in principle. He was questioning the measurability of the deliverables.

He laid out exactly what he expected: fixed prompts, before/after comparisons, and trackable share of voice. In other words, AI SEO should be run like a measurable experiment, not a random collection of conversation screenshots.

He mentioned coming across more rigorous approaches and specifically named tools like humanswith.ai and Profound as examples of what the industry should look like — "the standard now is tracking, not screenshots."

That's a fair assessment. Traditional SEO developed into a mature service market precisely because it has a quantifiable metrics framework: keyword rankings, organic traffic, click-through rates, conversions. AI visibility, as a new category, is fundamentally unverifiable if practitioners can't clearly articulate which prompts they're testing, how often they test, and what magnitude of change they're measuring.

Tools like Profound and humanswith.ai represent an "AI visibility tracking" capability whose core function is to systematically monitor a brand's Share of Voice in AI-generated responses. Share of Voice originated in traditional media ad monitoring, measuring a brand's exposure proportion on a given topic relative to competitors. Translated to the AI context, it typically means: across a predefined set of industry-relevant questions, the percentage of queries in which an AI model voluntarily mentions or recommends the brand. This mirrors the logic of "keyword rankings" in traditional SEO, but the unit shifts from "page position" to "mention rate in model responses." The value of these tools lies in providing repeatably sampled baseline data, making it possible to trace and verify the causal relationship between optimization actions (such as updating website content, increasing authoritative citations, or improving structured markup) and changes in visibility.

Why 'AI Visibility' Is Hard to Measure

To be fair, measuring AI SEO is genuinely more complex than traditional SEO — and that's fertile ground for confusion and abuse.

ChatGPT, Claude, Perplexity, and other large language models produce non-deterministic outputs. The same question asked at different times or in different contexts can yield different answers. This means a single screenshot carries almost no evidentiary weight — it only shows that "one model mentioned you once," says nothing about trends, and can't be attributed to any specific optimization action.

For this reason, any serious approach must include several elements:

• A fixed prompt set: Using a stable, reproducible set of questions queried repeatedly is the only way to make meaningful comparisons over time.
• Regular sampling: Running queries multiple times and averaging results offsets the randomness in model outputs.
• Share measurement: The question isn't "were we mentioned?" but "how has our citation/recommendation rate changed across relevant queries?"
• Timestamps and version logs: Models get updated. Screenshots without dates have no baseline value.

Without this methodology, any so-called "AI visibility report" is just a curated collection of screenshots — possibly cherry-picked from thousands of attempts to show the client the most favorable result.

It's worth noting that architectural differences across major AI products further compound measurement challenges. ChatGPT, Claude, Gemini, and others each have different training data cutoff dates, retrieval-augmented generation (RAG) strategies, and response generation mechanisms. Some products (like Perplexity) crawl web content in real time, so whether a brand gets cited depends heavily on the quality of pages indexed at that moment. Pure conversational models rely primarily on pre-training data, where brand recognition, authoritative media coverage, and structured data markup (like Schema.org annotations) carry more weight. This means "AI visibility" is actually a composite metric spanning multiple platforms and mechanisms — it cannot be represented by screenshots from a single model. Professional monitoring approaches typically need to cover multiple major AI products simultaneously, noting their respective version numbers or API snapshot dates. Without that, the data points simply can't be compared.

What Should $3,000 Extra Actually Buy?

Back to the poster's most practical concern: what should that additional $3,000 per month actually deliver?

A reasonable expectations checklist looks like this: a disclosed prompt testing set (the client has a right to know what's being tested), a clear baseline (visibility levels before any intervention), periodic tracking data (not a one-time snapshot), and a documented methodology. If an agency can't answer "which prompts are you testing?", you can reasonably conclude their "AI visibility" service hasn't developed into a real workflow yet — it looks more like a premium upsell riding the hype cycle.

The poster described himself as a client-side person rather than an agency veteran, and kept second-guessing himself with "maybe I'm missing something." But from a professional standpoint, his judgment is actually sharper than many practitioners in the field. In the explosive early window of a new concept, clients who dare to ask "show me the methodology and data" are exactly the ones least likely to get taken for a ride.

A Checklist for Buyers

For companies that are considering or have already purchased "AI SEO / AI visibility" services, this case offers several actionable evaluation criteria:

1. Ask for the prompt set: If they can't produce fixed test prompts, there's no reproducible measurement foundation.
2. Require a baseline and before/after comparison: Any optimization must have pre-intervention data as a reference point.
3. Be wary of screenshot-only deliverables: Screenshots can serve as supplementary illustrations, but they should never be the sole "proof of results."
4. Require dates and sampling frequency: Data without timestamps has no credibility.
5. Benchmark against professional tools: Profound, humanswith.ai, and similar platforms have made "tracking share of voice" their standard output format — use their deliverable structure as a yardstick to evaluate agencies.

The poster's question — "am I being played?" — is actually answered within his own observations. When an additional paid service cannot provide any measurable evidence, the paying party has every right to demand a methodology rather than accept a polished deck of slides. AI SEO is a real and growing trend — but being a trend doesn't mean it's exempt from scrutiny. Measurement standards are what separates professional service from marketing theater.