Brand Bias in Prompts: An Experiment

This experiment looked at three sets of prompts (100 each) — brand, “soft-brand”, and non-brand — all of them based on the topic “seo tools” and a handful of pre-selected brands. We intentionally kept the scope narrow, and within a domain we understood well. Of course, results may vary across different topic areas.

Brand prompts

This is the most straightforward group. Brand prompts contained a brand name or branded product directly in the prompt. Some examples include:

• “Can I see historical Domain Authority data in the Moz dashboard?”
• “How many domains does the Moz link index currently track?”
• “Is Moz or Semrush better for a beginner in SEO?”

Note that brand prompts could include brands or branded products and metrics.

Soft-brand prompts

The “non-brand” prompts were split into two groups. The soft-brand group used our query fan-out research to generate prompts in an open-ended way. Examples include:

• “Are premium search suites worth the investment for a small blog?”
• “Can I use a tool to find the most popular questions in my niche?”
• “How do I reconcile keyword scores from multiple search platforms?”

There’s a bias inherent in our topic — questions about seo tools are naturally going to include specific tools and brands in the answers. So, even without including a brand-name or biasing the system toward brands, we’ve already created a soft brand bias.

Non-brand prompts

Given the topic bias, we nudged the system to generate prompts that were more tool-adjacent, resulting in broader, informational questions. For example:

• “How do you measure the organic search visibility of a new website?”
• “Is it better to target one high-volume term or ten low-volume?”
• “What is the best way to handle a sudden drop in rankings?”

We’ll call these our true non-brand prompts. Even from these few examples, it’s probably clear that the line between non-brand and “soft-brand” is a gray one and depends a lot on the topic. Brand mentions are an on/off switch, but brand bias is a volume knob.