Data study · 1 edition

Do AI assistants know when a product changed?

Each run is saved as a dated edition and is never changed. Find the latest result, earlier editions and changes in the numbers here.

Latest edition: v1, October 2026

Data collected .

In the Claude pilot, web-search runs stated the verified change in 84 of 105 direct-question labels and 36 of 84 category-question labels. A label pairs an answer with a product; category totals count hedged claims but exclude neutral mentions and absent products. The verified change was stated in memory-only runs in 0 of 60 labels (neutral labels included) and in search runs on direct questions in 84 of 105 labels. For direct questions with search, the change was stated by Claude Opus 5.5 in 35 of 35 answers, Claude Fable 5.1 in 34 of 35, and Claude Sonnet 5.5 in 15 of 35 (model identifiers claude-opus-5-5, claude-fable-5-1 and claude-sonnet-5-5). Other assistants got the same questions in their browser interfaces, and their counts wait for a re-run that saves the full answers.

Read v1, October 2026

Key takeaways

  1. In the Claude pilot, web-search runs stated the verified change in 84 of 105 direct-question labels and 36 of 84 category-question labels. A label pairs an answer with a product; category totals count hedged claims but exclude neutral mentions and absent products.
  2. The verified change was stated in memory-only runs in 0 of 60 labels (neutral labels included) and in search runs on direct questions in 84 of 105 labels.
  3. For direct questions with search, the change was stated by Claude Opus 5.5 in 35 of 35 answers, Claude Fable 5.1 in 34 of 35, and Claude Sonnet 5.5 in 15 of 35 (model identifiers claude-opus-5-5, claude-fable-5-1 and claude-sonnet-5-5).
  4. A fresh reviewer read all 216 answers in full and labelled them, producing 270 answer-by-product labels; each label carries a verbatim quote checked by script, and four labels were spot-checked against the saved answers.
  5. Perplexity, Google AI Mode, ChatGPT and Gemini got the same questions on ; we will publish their counts only after a re-run that saves the full answers.

All editions

EditionData collectedPublishedHeadline numbers
v1, October 2026Direct-question labels 84 of 105; Category-question labels 36 of 84

Citing this study

Cite the edition page you used, not this series page:

Citing this study
CookiesForAI, "Do AI assistants know when a product changed?", v1, October 2026, https://cookiesforai.app/research/do-ai-assistants-know-a-product-changed-october-2026

CookiesForAI lists agent skills, MCP servers and apps, so our own pages can appear in these answers; when they do, we count them like any other site.