---
title: "Which Claude models tell you a product has changed?"
canonical_url: https://cookiesforai.app/benchmarks/claude-models-state-a-product-change
markdown: https://cookiesforai.app/benchmarks/claude-models-state-a-product-change.md
publisher: "CookiesForAI"
last_updated: 2026-10-25
type: article
summary: "The share of answers that stated the verified change, on direct questions about the product, with web search on. When you ask Claude by name about a coding tool that shut down, was renamed, was bought or changed its free plan, does the answer say so?"
---

> Full index of this site: https://cookiesforai.app/llms.txt  

# Which Claude models tell you a product has changed?

Updated 25 OCT 2026.

When you ask Claude by name about a coding tool that shut down, was renamed, was bought or changed its free plan, does the answer say so?

Only tests we ran ourselves, never what makers claim.

- 105 runs counted, one answer per run
- 3 models ranked

Where it ran: Claude Code, web search on.

What it measures: The share of answers that stated the verified change, on direct questions about the product, with web search on.

## The ranking

Higher is better.

Answers that stated the verified change, direct questions with web search, 04 OCT 2026.

| Rank | Name | Score | Runs | Model as shown | Date | Study |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | 35 of 35 (100%) | 35 | claude-opus-5-5 | 04 OCT 2026 | [Do AI assistants know when a product changed?](https://cookiesforai.app/research/do-ai-assistants-know-a-product-changed-october-2026) |
| 2 | Claude Fable 5.1 | 34 of 35 (97%) | 35 | claude-fable-5-1 | 04 OCT 2026 | [Do AI assistants know when a product changed?](https://cookiesforai.app/research/do-ai-assistants-know-a-product-changed-october-2026) |
| 3 | Claude Sonnet 5.5 | 15 of 35 (43%) | 35 | claude-sonnet-5-5 | 04 OCT 2026 | [Do AI assistants know when a product changed?](https://cookiesforai.app/research/do-ai-assistants-know-a-product-changed-october-2026) |

## What changed

- 25 OCT 2026: First edition, from the runs of 04 OCT 2026.

## All editions

- [v1, October 2026](https://cookiesforai.app/benchmarks/claude-models-state-a-product-change/2026-10-04): runs of 04 OCT 2026, published 25 OCT 2026

## Data and how to cite

Download the data: [CSV](https://cookiesforai.app/charts/do-ai-assistants-know-a-product-changed-october-2026.csv) (every chart of the study, this one included; v1, October 2026). Free to reuse with credit (CC BY 4.0).

How to cite: CookiesForAI, "Which Claude models tell you a product has changed?", benchmark, v1, October 2026, https://cookiesforai.app/benchmarks/claude-models-state-a-product-change/2026-10-04

## Questions people ask

### How do we choose what goes on a scoreboard?

Only tests we ran ourselves and published as a study, with the questions, the dates, the number of runs and the data. A board takes its numbers straight from the study's chart, so the two can never differ. We do not rank directories or other websites.

### Why can our numbers differ from what makers say?

Makers usually test their own setup, with their own questions. We type real questions into the chat apps in the browser, or into Claude Code with web search on, as anyone could, on a stated day, and count what we saw. A different day, account, model or wording can give a different count, which is why every board names where it ran and the day of the runs.

### What does "N of M" mean, and why do small samples matter?

M is how many runs we made and N is how many of them passed the test; each run gives one answer. The rank comes from N divided by M, and rows with the same share get the same rank. With a few dozen runs per row, a gap of one or two runs can be chance, so read close scores as close, not as a clear win.

### Do we have an interest of our own?

CookiesForAI lists agent skills, MCP servers and apps, so our own pages can appear in these answers; when they do, we count them like any other site.

### How do I send a correction?

Write to us with the board, the row and what you think is wrong. We check it against the saved runs and fix the board. [Contact us](https://cookiesforai.app/contact).

## Other benchmarks

- [Which AI assistants search the web before they answer?](https://cookiesforai.app/benchmarks/ai-assistants-search-before-answering)

[All benchmarks](https://cookiesforai.app/benchmarks) · [All studies](https://cookiesforai.app/research)
