AI Recommendations: How Product Information Drives Selection

Executive Summary

Why do ChatGPT, Claude, and Gemini suggest two popular lip treatments more than others?

Our latest research shows that slight copy changes that added information confirming specific user needs (in this case, that the product is suitable for users worried about fine lip lines) resulted in a significant, consistent increase in AI recommendations. Neutral copy produced no changes; adding “people who worry about fine lip lines” to the product description increased recommendations, but results were inconsistent across each model.

Target Product A received 0 of 90 controlled recommendations under control and 0 of 90 after neutral packaging copy. After inserting “fine lip lines” without connecting it to a brand name, it was selected in 47 of 90 recommendations (52.2%). Stating that Target Product A was suitable for fine lip lines resulted in 87 of 90 recommendations (96.7%). The percentage increased by 44.4 percentage points from keyword to user language, with a pooled paired comparison of p = 1.82 x 10^(-12).

The same outcome occurred with Target Product B, which went from 9 of 90 recommendations, or 10% under control, to 90 of 90, or 100%, after receiving the same type of direct suitability language. A pooled paired comparison showed p = 8.27 x 10^(-25). The controls show that, before any user language appeared in the product descriptions, the models could differentiate among products based on characteristics: products within the peptide group received 150/150 control recommendations, though the study does not have enough data to conclude whether the result was due to peptides or other factors.

The last condition was designed to show an advantage when two competing brands communicate similar product fit. When Target Product A and Target Product B were given identical user-need statements, they received 49 of 90 recommendations (54.4%) and 39 of 90 recommendations (43.3%), respectively. Together, Target Product A and Target Product B received 88 of 90 recommendations (97.8%), leaving Peptide Comparator with 2.2%.

The study supports the idea that AI models can infer relevance from product information, and keyword overlap may affect the number of recommendations generated; however, language that connects the product with a user’s need produces consistent AI recommendations.

How Does Specific Product Information Affect Peptide Treatment Recommendations?

Product display pages (PDPs) are full of ingredients, formulas, features, and product categories. However, they rarely explain the specific situation or problem that makes a product a viable alternative. Our recent study examined how product description phrasing influences recommendations from ChatGPT, Claude, and Gemini for users seeking lip treatments for dry lips and fine lines.

We built on earlier 5WPR research testing an occasion-based request for rosé wine. In this follow-up study, we observed how product information and its use would affect AI recommendations for a user looking to buy a hydrating lip treatment to address fine lines.

In this study, we compared eight different lip treatments. We hid each brand’s identity, gave all eight brands equal standard product information, ran each competitor’s trial through three popular models, held treatment conditions constant, and used paired statistical analysis to determine whether the models’ recommendation rates differed significantly between conditions.

The study design revolved around these three questions:

• Does adding an extra sentence improve the recommendation rate for a product?

• Does using the exact language in the user’s query improve AI recommendation rates?

• Does stating the product is suitable for the user’s concern produce a different recommendation rate than either of the two questions above?

These findings allow us to measure the impact of content on product display pages across earned media and websites.

Can Models Recommend Peptide Treatments Without User-Need Information?

Yes. The untreated results indicate that the models could distinguish among the products before any user language was presented.

Control condition: the models separated products before any user language appeared.

Within two separate control samples, products within the peptide group were selected 100 percent of the time (150/150). The four non-peptide products weren’t selected by any model.

One peptide product accounted for most of that total. Peptide Comparator, which was not one of the two products under test, received 91 of the 150 control recommendations, or 60.7%. In the 90-trial control sample used for the comparisons in this study, it received 59 recommendations, or 65.6%, and it was the most frequently selected product for all three models. Neither Target Product A nor Target Product B led the field before we added any user language.

The product descriptions included information regarding ingredients, product type, and formula. None of these descriptions indicated that the product was suited for individuals with fine lip lines.

We can’t conclude that peptides drove the preference because the formula, product type, and ingredient combinations differ significantly among products, but we learned that the information provided was sufficient for the models to distinguish among the available products.

Next, we tested whether letting the models infer the user-need relationship, rather than stating it explicitly, would produce a similar outcome.

How Did the Peptide Treatment Study Test Keywords, Added Copy, and User Language?

How did the test distinguish between keywords and the product relationship fit?

We modified one sentence and left the remainder of the product information identical.

The four conditions were

• Control: no additional sentences were added to the original product descriptions.

• Neutral copy: “[Product] comes in a labeled container with the product name on the package.”

• Keyword only: “Fine lip lines are a common concern discussed in lip-care routines.”

• Stated user-need: “[Product] is suitable for people concerned about fine lip lines.”

Each condition served a different function.

The neutral copy condition tested whether adding copy alone would increase recommendations.

The keyword-only condition used the same language that consumers typically mention in their need/concerns statements but did not associate it with any product.

The final condition tied the user’s concern to one choice.

The findings are important for content development strategies because adding copy, matching user-generated query language, and explaining how a product fits a user’s need are not interchangeable functions.

AI models often eliminate brands whose content or PDP includes exact-match phrases but don’t address a user’s need.

What Increased Target Product A’s Recommendation Rate From 0% to 96.7%?

A clear sequence of events drove Target Product A’s recommendations.

Target Product A’s recommendation rate across all four conditions.

Target Product A began at 0% in the control group.

The neutral packaging copy held Target Product A’s recommendation rate at 0%.

Adding “fine lip lines” without connecting it with Target Product A increased its recommendation rate to 52.2%.

Directly tying the concern to Target Product A increased its recommendation rate to 96.7%.

The latter increase was 44.4 percentage points greater than Target Product A’s recommendation rate with keyword-only language.

The pooled comparison resulted in p = 1.82 × 10⁻¹².

How Do Keywords Affect Peptide Treatment Recommendations?

Yes. While the keyword-only condition produced a large effect relative to control, the magnitude differed substantially between models.

Keyword-only results ranged from 20% to 86.7%. User-need language ranged from 90% to 100%.

These results show the difference between discussing a user’s needs and clearly communicating how a product addresses them to help the user determine if a suitable option exists.

Did the Finding Replicate a Similar Peptide Treatment?

Did the study’s result replicate with Target Product B?

Yes, Target Product B provided an additional test with a different initial position.

Target Product B moved from 9 of 90 recommendations under control to 90 of 90 with stated user-need language.

Target Product B received 9 of 90 recommendations, or 10%, in the control group.

When Target Product B was given the same copy as Target Product A’s, ChatGPT, Claude, and Gemini each recommended it 30 out of 30 times.

Combined results totaled 90/90 (100%). The increase was ninety percentage points, with a pooled comparison resulting in p = 8.27 × 10⁻²⁵.

Unlike Target Product A, whose initial position was 0%, Target Product B initially received a recommendation share in the control because it was part of the peptide group.

However, both Target Product A and Target Product B achieved near-perfect recommendation rates when evaluated separately with suitability language (mentioning the benefit for fine lines) applied to each product.

What Happened When Peptide and Non-Peptide Treatments Made the Same Claim?

The individual advantages decreased when both competitors communicated comparable claims at the same time.

Results changed when Target Product A and Target Product B received comparable language simultaneously:

Recommendation rates when both brands conveyed the same user/product fit.

The results suggest that information that distinguishes one qualified product becomes competitive space when another qualified option conveys the same user/product fit. Notably, Peptide Comparator received 65.6% of the recommendations in the control group and 2.2% when Target Product A and Target Product B stated the same user need.

No conclusions should be drawn regarding each category producing an analogous result. The 97.8% value represents this controlled test and should not be interpreted as a universally applicable benchmark.

How Should Brands Apply These Findings?

This study offers actionable insights for developing and improving product content.

Five ways to apply the findings to product and brand content.

Identify consumer needs and concerns that drive product comparison decisions.

Develop any suitability statements supported by evidence demonstrating product properties.

Do not aggregate multiple changes (keyword additions, new claims, additional copy) to identify which variable(s) contributed to movement.

Differences among model responses can be significant enough to lead to opposite conclusions based on small sample sizes.

Be aware of category differences; advantages from informative content may diminish as competitors improve their product content.

AI models can draw inferences from product content; however, those inferences shouldn’t be assumed. Future studies will evaluate this method in other categories, compare different forms of suitability language, and assess whether similar improvements occur when models gather information from online webpages rather than controlled product lists.

Turn Peptide Product Information Into Consistent AI Recommendations

When products are relevant to common user needs, but AI models aren’t recommending your brand, the problem may be a gap in how you communicate product fit.

5W helps brands identify relevant user needs, audit product and brand content for the missing relationships that AI models look for, test how different types of information affect model recommendations, and build content strategies designed to improve visibility and conversions across AI search environments.

Contact 5W to measure and improve presence in AI-generated recommendations.

Methodology

This experiment studied how different forms of product information affect AI model recommendations for 8 commercial lip care products.

The user request was:
“I have fine lip lines. Can you tell me what one product you think would best moisturize them and help make those fine lines look smoother over time?”

Each model was given a single recommendation based only on the product information presented to it.

Two target products were chosen for the study:

Target Product B, which represented the peptide group,
And Target Product A, which represented the non-peptide group.

Six additional products made up a competitive candidate set containing lip treatments, balms, masks and an ointment.

Information for each product was taken from official brand product pages (captured before testing) and then reviewed.

In addition, every product description had comparable information:
Product type: lip balm, lip mask, lip treatment etc.
Key ingredients: up to three standardized ingredient names as shown in the brands’ product information.
Formula/format: a short description of the balm, mask, ointment or glaze formula.

No claims of effectiveness, clinical data percentages, prices or reviews were included in the descriptions to prevent these items from answering the fine line question independently.

Examples of product descriptions include:

The Target Product A description read:
Product type: lip balm
Key ingredients: Shea Butter; Murumuru seed butter; Sodium Hyaluronate
Formula/format: Buttery balm formula.

The Target Product B description read:
Product type: lip treatment
Key ingredients: Palmitoyl tripeptide-1; Shea Butter; tocopherol
Formula/format: thick glaze formula.

Ingredient fields were limited to 3 to ensure that product descriptions maintained structural comparability.

Comparator Product D had fewer ingredient listings since its official formulation lists only two.

The models were “pinned” versions of models and thus did not automatically update to “the latest version”:

Platform Model
ChatGPT GPT-4.1-2025-04-14
Claude Claude Opus 4.5, 20251101
Gemini Gemini 3.5 flash

Web searching was disabled.

The temperature was set to 0.0.

The system instruction advised each model to rely solely on the information it received regarding the Products, to pick one candidate and to provide its choice in a predetermined JSON format.

All product identifiers were anonymized.

We replaced product brands and product names with generic aliases.

Candidate order and aliases were randomly assigned but consistent throughout each pair of trials.

Consequently, a treatment condition could be compared against its corresponding control condition with the same presentation.

The randomization of aliases eliminated the possibility that a result could be since a product appeared at a different location or with a different alias between the treatment and control conditions.

The anonymity referred to names, not formulations. Therefore, distinctively formulated products remained identifiable because the model could still see their ingredients.

The study compared four different variations of treatment conditions.

Treatment condition definition
Control added no new sentence.
Neutral copy “[product] comes in a labeled container with the product name on the package.”
keyword only “fine lip lines are a common concern discussed in lip-care routines.” This sentence contained the exact phrase “fine lip lines,” but did not link it to either of the target Products.
Direct suitability “[product] is suitable for people who are concerned about fine lip lines.”

To test whether having a competitor present with the user-need statement would affect performance, the statement was tested simultaneously with Target Product A and Target Product B.

The study conducted 540 successful calls to the models via DataforSEO.

There were 30 trials per model for each of the six conditions or 90 recommendations for each model across ChatGPT, Claude and Gemini.

The conditions were:

Calls
Shared control 90
Target Product A direct suitability 90
Target Product A neutral copy 90
Target Product A keyword only 90
Target Product B direct suitability 90
Both Target Product A and Target Product B direct suitability 90
Total 540

The conditions utilized paired trial plans so candidate order and anonymous aliases were consistent across relevant comparisons.

Data collected during included:

0 API errors.
0 parse errors.
0 multiple recommendation errors.

Was there a manipulation check?

Yes.

Before analyzing the primary data, a manipulation check ran 20 untreated trials per model, for a total of 60 calls.

These trials used a unique seed and were excluded from all analyses, regardless of outcome.

The manipulation check revealed an interesting baseline trend: all 60 recommendations went to products in the peptide group, while all 4 non-peptide products received zero.

Another control sample was run later using the primary protocol that demonstrated the group level trend – i.e., 90 peptide group selections out of 90 total recommendations.

Thus, together, the two control samples yielded 150 peptide group recommendations out of 150 possible recommendations.

How were model responses evaluated?

Model responses were evaluated using deterministic name matches rather than asking another model to determine whether the answer was correct.

System responses (i.e., raw model responses) were stored before being parsed into usable output.

Because each trial consisted of two conditions (i.e., each trial was paired), we used paired comparisons instead of independent comparisons to compare the target between conditions.

Paired comparisons used McNemar tests to determine whether selecting/unselecting a target differed between conditions.

If the p-value associated with an interaction was < 0.05, it was assumed that there existed a statistically significant effect of a condition relative to a reference condition.

Primary pooled comparisons included:

Comparative test Target Product A keyword only vs. Control p = 1.42 × 10−14
Comparative test Target Product A suitable vs. keyword only p = 1.82 × 10−12
Comparative test Target Product B suitable vs. Control p = 8.27 × 10−25

Study Limitations

We did not test live searching

Although we supplied product information directly to the models in frozen form, we did not attempt to simulate how this information might arrive via live crawling/searching.

Therefore, we cannot conclude that adding one sentence to a live product page will cause the same percent change.

Blinding eliminated brand/product name cues, not ingredient cues

While brand/product names were obscured by anonymity, none of the ingredients in the formulation were anonymous.

Models trained on formulations could infer which product was behind an anonymous alias, e.g., Peptide Comparator’s argan ingredient is unique among the eight Products studied. Similarly, Comparator Product D and other products contain unique ingredient combinations that can be used as identification cues.

Thus, while anonymity was achieved for names, it was not achieved for formulations.

The study was restricted to one user request

one single request concerning dry lips and fine lip lines was tested in this study.

Variations of wording, varying concerns or varying emphasis on a different characteristic could yield different ranked products in future studies concerning lip care recommendations.

Findings should be replicated across additional requests before being generalized to lip care recommendations as a whole

The eight product candidate set was predefined Every measurement reported above resulted from competition among these products.

Adding or deleting candidates could alter recommendation rates. The competition test depends heavily on the candidate set because each model had to pick exactly one product from the available options.

Therefore, the concentration of approximately 98% for Target Product A and Target Product B must be viewed in this context and cannot be used as a benchmark for other product categories.

The study examined three versions of AI models

Chatgpt, Claude and Gemini varied in terms of their response to “keyword-only” language.

Future versions of these models may differ from what was seen in this study.

This study demonstrates behavior of tested model versions under tested conditions and does not demonstrate enduring behavior for the platforms themselves.

The study did not examine durability of recommendation effects

This study measured model recommendations during a specified testing window.

However, this study did not examine if this advantage would continue after model revisions, changes to competing product descriptions or changes in retrieval algorithms.

Long-term testing would be needed to assess durability.

Recommendation effects do not represent efficacy data regarding product performance

This study examined which product a model recommended.

This study did not evaluate whether a particular lip treatment improved appearance of fine lines; whether certain formulations performed better than others; or whether claims regarding marketing adequacy are supported by sufficient clinical or regulatory data.

Brands using these findings should make marketing claims they can independently support.