AI-Generated Brand Recommendations Are Exposing a Measurement Problem

New research on AI-generated brand recommendations is a canary in the coal mine for how AI is reshaping marketing measurement in real time.

Rand Fishkin of Sparktoro just published an analysis showing that AI tools, including LLM-based systems, are wildly inconsistent when generating AI-generated brand recommendations. Identical prompts, run repeatedly through the same model, produce different brand lists, different ordering, and different counts. What looks like “ranking” is often statistical noise rather than a stable outcome.

For brands and agencies, this should be unsettling. Many of the signals we’ve historically relied on – SERP rank, search share of voice, website visits – no longer tell the full story. And unlike classic search, AI-generated answers don’t come with clean, inspectable placement data that can be audited or tracked over time.

Why AI Visibility Doesn’t Behave Like Search

There’s a second layer that compounds the challenge.

People don’t ask questions the same way, and increasingly AI systems may have contextual knowledge about the person asking. Preferences, interests, constraints, and conversational cues all influence AI-generated brand recommendations and which brands are surfaced.

That means there is no single “AI result” for a brand to optimize against. There are millions of possible answer paths, shaped by user behavior, system inference, and interaction design.

From Ranking to Consideration

This shift forces a change in how success is defined.

In LLM-mediated discovery, the objective isn’t to move up in a fixed list. It’s to be included in the system’s consideration set across many contexts and users. Brands that appear consistently within AI-generated brand recommendations outperform those that optimize for isolated wins in individual responses.

Visibility without consideration is functionally invisible.

This isn’t the first time flawed measurement frameworks have distorted outcomes. Similar dynamics show up in procurement and pitch processes, where poorly designed criteria reward the wrong behaviors and create false confidence.

What Leaders Should Do Now

The implications of AI-generated brand recommendations aren’t tactical. They’re structural. That means the response has to start at the leadership level.

• Reset expectations around AI visibility. AI-generated brand recommendations should be treated as probabilistic signals, not ranked outcomes. Individual responses vary by user, context, and interaction. Stop equating position in a single AI answer with performance, and resist the urge to overinterpret isolated results.
• Reframe success from “ranking” to “being considered.” In LLM-mediated discovery, the strategic objective isn’t to move up a fixed list. It’s to be included consistently in the AI’s consideration set across many contexts and users. Brands that show up reliably within AI-generated brand recommendations outperform those optimizing for one-off wins.
• Make your brand legible to machines. There is meaningful low-hanging fruit here. Structured data, clean entity definitions, and emerging protocols like MCP help AI systems reliably understand, retrieve, and trust authoritative information about your brand and products. This work isn’t glamorous, but it’s foundational.

Why This Matters

This is a shift that demands decisive leadership. It’s time to establish clear governance across marketing, revenue, and agency partners to align incentives, metrics, and decision-making as AI becomes a core discovery layer.

Photo by Jelle Taman on Unsplash