Data study · v1, October 2026

How ChatGPT, Gemini, Perplexity, Google and Grok answer software and AI tool questions

All editions of this study

The short answer

Most of the six AI assistants we tested searched the web before naming software: ChatGPT showed web citations in 21 of 21 main runs, Perplexity searched in 11 of 11, Gemini 3.6 Flash showed sources in 14 of 21 runs and Grok in 18 of 21, and Google AI Mode and AI Overviews were search-grounded in all 34 runs we logged (21 AI Mode, 13 AI Overviews). For broad questions the same leaders came first on every engine: There's An AI For That was named first for the best AI tools directory in the first run on all six engines. For narrow agent-skill questions, the sources behind the answers were small skill directories (LobeHub in 5 runs, MCP Market in 4), and the picks changed from engine to engine and from run to run. These numbers come from 110 runs on .

Key takeaways

  1. ChatGPT showed web citations in 21 of 21 main runs and Perplexity searched in 11 of 11; Gemini 3.6 Flash showed sources in 14 of 21 runs and Grok in 18 of 21.
  2. There's An AI For That came first for the best AI tools directory question in the first run on all 6 engines.
  3. For three agent-skill questions, the sources behind the answers were small skill directories: LobeHub in 5 runs, MCP Market in 4, agentmods in 3 and skills.rest in 3.
  4. ChatGPT gave 3 different first picks in 3 runs of the same Terraform skill question.
  5. Perplexity searched the exact question in 10 of 11 runs and named products in the order of the pages that ranked.
  6. ChatGPT put OpenAI Codex first in 3 of 3 runs on new AI coding tools.

We asked six AI assistants the same 13 questions people ask about software and AI tools, on , and recorded every product they named, in order, and every source they showed. Across 110 runs, the head of each answer was stable and shared by all engines, while answers to narrow questions about agent skills changed from engine to engine and from run to run, and were decided by small directory pages.

What did we find?

  1. Search is the default. ChatGPT cited the web in 21 of 21 main runs, Perplexity searched in 11 of 11 and Google AI Mode and AI Overviews were search-grounded in all 34 runs we logged. Gemini showed sources in 14 of 21 runs (on Gemini 3.6 Flash), Grok in 18 of 21.
  2. The same names lead everywhere. For "What is the best AI tools directory for finding new AI tools?", There's An AI For That came first in the first run on all six engines (ChatGPT later moved Product Hunt and Futurepedia to first in two of its other runs). AFFiNE and AppFlowy led every Notion-alternatives answer; Exa, Firecrawl, Brave and Tavily every MCP-for-web-research answer.
  3. Narrow skill questions are a lottery, decided by skill directories. For three agent-skill questions, the sources that decided the answer were small skill directories (LobeHub in 5 runs, MCP Market in 4, agentmods in 3 and skills.rest in 3). ChatGPT gave three different "best fit" skills in three runs of the same question.
  4. Perplexity follows its top search result. It searched the exact question in 10 of 11 runs and named products in the order of the pages that ranked; a vendor's own "best X" list put that vendor first (KIME, Firecrawl).
  5. Wording opens the long tail. On Google AI Mode, adding "open source" replaced the whole directory list; "for beginners" changed the first pick. Asking ChatGPT "Any smaller or newer ones?" brought 7 directories no first answer had named.
  6. Engines route to their own products. ChatGPT put OpenAI Codex first in 3 of 3 runs on new coding tools; Gemini and Grok each answered one skill question with their own built-in skills.

Which skill directories were behind the agent-skill answers?

  1. LobeHub5
  2. MCP Market4
  3. agentmods3
  4. skills.rest3

Runs in which each small skill directory was a source behind the answer to one of the three agent-skill questions, across all engines, .

Source: CookiesForAI, v1, October 2026

Reuse this chart

Free to reuse with credit (CC BY 4.0). Download SVG

Embed code
<figure><img src="https://cookiesforai.app/charts/ai-assistants-field-study-october-2026--skill-directories.svg" alt="Which skill directories were behind the agent-skill answers: LobeHub 5 runs, MCP Market 4 runs, agentmods 3 runs, skills.rest 3 runs."><figcaption>Source: CookiesForAI, Which skill directories were behind the agent-skill answers?, https://cookiesforai.app/research/ai-assistants-field-study-october-2026</figcaption></figure>

How did we run the test?

Date and place , 08:45 to 13:45 Berlin time, from a Munich IP address
Engines ChatGPT (page model gpt-5-6, then gpt-5-6-mini), Gemini 3.6 Flash, Perplexity (Search mode, model not shown), Google AI Mode, Google AI Overviews, Grok (Fast)
Not run Copilot, DeepSeek and Kimi asked for a login before answering
Questions 13, typed exactly as below, each in a fresh private or temporary chat
Repeats Questions 5, 6, 7 and 11 three times on ChatGPT, Gemini, Google AI Mode and Grok
Runs 110: ChatGPT 23, Gemini 21, Google AI Mode 21, Grok 21, Google AI Overviews 13, Perplexity 11
How A browser agent typed each question in the engines' own web apps and logged names, links, source panels and visible search queries

All 13 questions are listed below, in order, question 1 first.

Which names came first for each question in run 1?

Q ChatGPT Gemini Perplexity Google AI Mode Grok
4 Notion alternatives AFFiNE, AppFlowy, Outline AppFlowy, AFFiNE, Anytype AFFiNE, AppFlowy, Docmost AppFlowy, Docmost, AFFiNE AFFiNE, AppFlowy, Anytype
6 MCP for web research Firecrawl, Exa, Tavily Exa, Brave, Tavily Firecrawl, Parallel Search, Exa Exa, Tavily, Firecrawl Exa, Brave, Tavily
7 Best AI tools directory TAAFT, Toolify, Futurepedia TAAFT, Futurepedia, Product Hunt TAAFT, Product Hunt, Futurepedia TAAFT, Futurepedia, Toolify TAAFT, Product Hunt, Futurepedia
9 Widest categories Toolify, Futurepedia, TAAFT TAAFT, Futurepedia, Toolify AIChief, Toolify, TopAI.tools Toolify, TAAFT, AIChief Toolify, TAAFT, Futurepedia
11 Terraform plan review skill terraform-review (Knackbox), terraform-plan-review, HashiCorp terraform-skill (Anton Babenko), a custom skill terraform-plan-review, plan-analyzer, a custom SKILL.md HashiCorp skills, Babenko terraform-skill, Terramate terraform-plan-review, oma-tf-infra, Babenko terraform-skill
12 Video with code Remotion Agent Skills, Video Shotcraft, HyperFrames remotion-best-practices, hyperframes, p5.js not run Remotion, HyperFrames, Framewright ffmpeg (Grok's own), imagemagick
13 Database migrations migration-safety, database-migrations (two repos) none, offered to build one not run two community database-migrations skills write-safe-migrations, database-migrations

TAAFT is There's An AI For That. Perplexity reached its free search limit after question 11.

How did each engine answer?

ChatGPT cited small listicle and blog pages most (anewera.ai, marqeable.com, fast.io, stackbuilt.co, siteefy.com, ossalt.com), plus vendor sites and GitHub. It never showed its search queries. Its long-tail answers were the least stable: three runs of question 11 gave three different first picks. In a Temporary chat set to "Personalized", details from the account's memory appeared in answers twice; a rerun set to "Unpersonalized" showed none.

Gemini 3.6 Flash answered without visible sources in 7 of 21 runs and was the most stable engine: the same sources and order in all three runs of question 7, and the same first pick in all three runs of question 11. For question 13 it looked for skills in the user's own Gemini workspace and offered to build one, naming no third-party skill.

Perplexity showed its queries, which were the exact question in 10 of 11 runs, and named products in the order of the top results. It surfaced the most small names (AITopTools, AIChief, AI Tool Tips, ToolsPedia).

Google AI Mode and AI Overviews cited Reddit and YouTube in almost every answer, plus the same handful of directory listicles. The two disagreed on the same question: for question 1, AI Mode's first pick was Radar Kit and the AI Overview's was Profound.

Grok showed its queries, which often already contained brand names from memory ("best AI search monitoring tool SaaS mentions Profound Peec Otterly 2026"). It was the only engine to check per-seat pricing for question 4, and it answered question 12 from its own bundled ffmpeg skill.

What does this mean for anyone listing a product or a skill?

  • For broad questions, the same few leaders were among the first names on every engine. A new product enters through narrow questions, a qualifier in the question, or a follow-up.
  • For narrow questions, one page per item on a directory is what the engines found and used. That page's title and first lines matter most, because engines often answer from search results without opening the page.
  • Several engines treated listings on third-party registries as proof that a product is real: asked "Is Launchelion legit?", ChatGPT cited its MCP server's listing on Glama and its Chrome Web Store page.

What are the limits?

One day, one location, free accounts. Most engines ran logged in to a neutral test account; Grok ran in a private chat on a personal account. ChatGPT's Temporary chats were probably personalized for most runs. Model names are as the pages showed them and were not verified. Answers churn: treat each count as what happened on , not as a rule. We will repeat the study and date each edition.

The questions we asked

  1. What's the best tool to check whether ChatGPT, Perplexity and other AI assistants mention my SaaS?
  2. I just launched a small SaaS. Where should I list it so AI assistants start recommending it?
  3. What agent skills should I install so my coding agent can work with PDFs and spreadsheets?
  4. What are good alternatives to Notion for a 3-person team on a tiny budget, ideally built by indie developers?
  5. What new AI coding tools launched this month that are worth trying?
  6. Which MCP servers should I connect to my AI agent for web research?
  7. What is the best AI tools directory for finding new AI tools?
  8. Which AI tools directory includes user reviews and ratings?
  9. Which AI tools directory has the widest range of categories?
  10. How do I post to Instagram from Claude?
  11. Is there an agent skill that helps my coding agent review Terraform plans?
  12. What agent skills exist for making videos with code?
  13. Find me an agent skill for writing database migrations safely.

Data and how to cite

Download the chart data: CSV, JSON. Free to reuse with credit (CC BY 4.0).

How to cite
CookiesForAI, "How ChatGPT, Gemini, Perplexity, Google and Grok answer software and AI tool questions", v1, October 2026, https://cookiesforai.app/research/ai-assistants-field-study-october-2026

CookiesForAI lists agent skills, MCP servers and apps, so our own pages can appear in these answers; when they do, we count them like any other site.

Questions people ask

Do AI assistants search the web before recommending software?

Mostly yes, in our October 2026 test. ChatGPT showed web citations in 21 of 21 main runs, Perplexity searched in 11 of 11, and Google AI Mode and AI Overviews were search-grounded in all 34 runs we logged. Gemini 3.6 Flash showed sources in 14 of 21 runs and Grok searched visibly in 18 of 21.

Which AI tools directory do AI assistants recommend?

There's An AI For That was named first for "What is the best AI tools directory for finding new AI tools?" in the first run on all six engines we tested, and in 36 of 36 runs of an earlier test on three Claude models. Futurepedia and Toolify followed on most engines.

Do AI assistants recommend their own company's products?

Sometimes. ChatGPT put OpenAI Codex first in 3 of 3 runs of "What new AI coding tools launched this month?". Gemini answered a request for a database-migration skill with its own Skills feature, and Grok answered a video skill question with its own bundled ffmpeg skill. Google added Gemini CLI to the coding tools answer.

Changes to this page

  • : short answer, takeaways, charts and the question list added; section headings now ask questions.
  • : ChatGPT's search count now uses its 21 main runs, like Gemini and Grok (21 of 21; it said 22 of 22, counting one of its two Unpersonalized control runs). Google's grounding is now stated as what we logged.