--- title: "How 6 AI assistants answer AI tool questions: 110 runs, October 2026" canonical_url: https://cookiesforai.app/research/ai-assistants-field-study-october-2026 last_updated: 2026-10-01 type: article summary: "We asked ChatGPT, Gemini, Perplexity, Google AI Mode, Google AI Overviews and Grok the same 13 questions about software and AI tools, 110 times on 2026-10-01, and recorded every name and source they gave." --- # How ChatGPT, Gemini, Perplexity, Google and Grok answer software and AI tool questions Updated 2026-10-01. First published 2026-10-01. We asked six AI assistants the same 13 questions people ask about software and AI tools, on Thursday 1 October 2026, and recorded every product they named, in order, and every source they showed. Across 110 runs, the head of each answer was stable and shared by all engines, while answers to narrow questions about agent skills changed from engine to engine and from run to run, and were decided by small directory pages. ## What we found, in six lines 1. **Search is the default.** ChatGPT showed web citations in 22 of 22 logged runs, Perplexity searched in 11 of 11 and Google always grounds in search. Gemini 3.6 Flash showed sources in 14 of 21 runs, Grok in 18 of 21. 2. **The same names lead everywhere.** For "What is the best AI tools directory for finding new AI tools?", There's An AI For That came first in the first run on all six engines (ChatGPT later moved Product Hunt and Futurepedia to first in two of its other runs). AFFiNE and AppFlowy led every Notion-alternatives answer; Exa, Firecrawl, Brave and Tavily every MCP-for-web-research answer. 3. **Narrow skill questions are a lottery, decided by skill directories.** For three agent-skill questions, the sources that decided the answer were small skill directories (LobeHub in 5 runs, MCP Market in 4, agentmods and skills.rest in 3 each). ChatGPT gave three different "best fit" skills in three runs of the same question. 4. **Perplexity follows its top search result.** It searched the exact question in 10 of 11 runs and named products in the order of the pages that ranked; a vendor's own "best X" list put that vendor first (KIME, Firecrawl). 5. **Wording opens the long tail.** On Google AI Mode, adding "open source" replaced the whole directory list; "for beginners" changed the first pick. Asking ChatGPT "Any smaller or newer ones?" brought 7 directories no first answer had named. 6. **Engines route to their own products.** ChatGPT put OpenAI Codex first in 3 of 3 runs on new coding tools; Gemini and Grok each answered one skill question with their own built-in skills. ## How we ran the test | | | |---|---| | Date and place | 2026-10-01, 08:45 to 13:45 Berlin time, from a Munich IP address | | Engines | ChatGPT (page model gpt-5-6, then gpt-5-6-mini), Gemini 3.6 Flash, Perplexity (Search mode, model not shown), Google AI Mode, Google AI Overviews, Grok (Fast) | | Not run | Copilot, DeepSeek and Kimi asked for a login before answering | | Questions | 13, typed exactly as below, each in a fresh private or temporary chat | | Repeats | Questions 5, 6, 7 and 11 three times on ChatGPT, Gemini, Google AI Mode and Grok | | Runs | 110: ChatGPT 23, Gemini 21, Google AI Mode 21, Grok 21, Google AI Overviews 13, Perplexity 11 | | How | A browser agent typed each question in the engines' own web apps and logged names, links, source panels and visible search queries | The 13 questions: 1. What's the best tool to check whether ChatGPT, Perplexity and other AI assistants mention my SaaS? 2. I just launched a small SaaS. Where should I list it so AI assistants start recommending it? 3. What agent skills should I install so my coding agent can work with PDFs and spreadsheets? 4. What are good alternatives to Notion for a 3-person team on a tiny budget, ideally built by indie developers? 5. What new AI coding tools launched this month that are worth trying? 6. Which MCP servers should I connect to my AI agent for web research? 7. What is the best AI tools directory for finding new AI tools? 8. Which AI tools directory includes user reviews and ratings? 9. Which AI tools directory has the widest range of categories? 10. How do I post to Instagram from Claude? 11. Is there an agent skill that helps my coding agent review Terraform plans? 12. What agent skills exist for making videos with code? 13. Find me an agent skill for writing database migrations safely. ## First three names per question (run 1) | Q | ChatGPT | Gemini | Perplexity | Google AI Mode | Grok | |---|---|---|---|---|---| | 4 Notion alternatives | AFFiNE, AppFlowy, Outline | AppFlowy, AFFiNE, Anytype | AFFiNE, AppFlowy, Docmost | AppFlowy, Docmost, AFFiNE | AFFiNE, AppFlowy, Anytype | | 6 MCP for web research | Firecrawl, Exa, Tavily | Exa, Brave, Tavily | Firecrawl, Parallel Search, Exa | Exa, Tavily, Firecrawl | Exa, Brave, Tavily | | 7 Best AI tools directory | TAAFT, Toolify, Futurepedia | TAAFT, Futurepedia, Product Hunt | TAAFT, Product Hunt, Futurepedia | TAAFT, Futurepedia, Toolify | TAAFT, Product Hunt, Futurepedia | | 9 Widest categories | Toolify, Futurepedia, TAAFT | TAAFT, Futurepedia, Toolify | AIChief, Toolify, TopAI.tools | Toolify, TAAFT, AIChief | Toolify, TAAFT, Futurepedia | | 11 Terraform plan review skill | terraform-review (Knackbox), terraform-plan-review, HashiCorp | terraform-skill (Anton Babenko), a custom skill | terraform-plan-review, plan-analyzer, a custom SKILL.md | HashiCorp skills, Babenko terraform-skill, Terramate | terraform-plan-review, oma-tf-infra, Babenko terraform-skill | | 12 Video with code | Remotion Agent Skills, Video Shotcraft, HyperFrames | remotion-best-practices, hyperframes, p5.js | not run | Remotion, HyperFrames, Framewright | ffmpeg (Grok's own), imagemagick | | 13 Database migrations | migration-safety, database-migrations (two repos) | none, offered to build one | not run | two community database-migrations skills | write-safe-migrations, database-migrations | TAAFT is There's An AI For That. Perplexity reached its free search limit after question 11. ## Findings per engine **ChatGPT** cited small listicle and blog pages most (anewera.ai, marqeable.com, fast.io, stackbuilt.co, siteefy.com, ossalt.com), plus vendor sites and GitHub. It never showed its search queries. Its long-tail answers were the least stable: three runs of question 11 gave three different first picks. In a Temporary chat set to "Personalized", details from the account's memory appeared in answers twice; a rerun set to "Unpersonalized" showed none. **Gemini 3.6 Flash** answered without visible sources in 7 of 21 runs and was the most stable engine: the same sources and order in all three runs of question 7, and the same first pick in all three runs of question 11. For question 13 it looked for skills in the user's own Gemini workspace and offered to build one, naming no third-party skill. **Perplexity** showed its queries, which were the exact question in 10 of 11 runs, and named products in the order of the top results. It surfaced the most small names (AITopTools, AIChief, AI Tool Tips, ToolsPedia). **Google AI Mode and AI Overviews** cited Reddit and YouTube in almost every answer, plus the same handful of directory listicles. The two disagreed on the same question: for question 1, AI Mode's first pick was Radar Kit and the AI Overview's was Profound. **Grok** showed its queries, which often already contained brand names from memory ("best AI search monitoring tool SaaS mentions Profound Peec Otterly 2026"). It was the only engine to check per-seat pricing for question 4, and it answered question 12 from its own bundled ffmpeg skill. ## What this means for anyone listing a product or a skill - Broad questions are decided before the search: the leaders are named from memory on every engine. A new product enters through narrow questions, a qualifier in the question, or a follow-up. - For narrow questions, one page per item on a directory is what the engines found and used. That page's title and first lines matter most, because engines often answer from search results without opening the page. - Several engines treated listings on third-party registries as proof that a product is real: asked "Is Launchelion legit?", ChatGPT cited its MCP server's listing on Glama and its Chrome Web Store page. ## Limits of this study One day, one location, free accounts. Most engines ran logged in to a neutral test account; Grok ran in a private chat on a personal account. ChatGPT's Temporary chats were probably personalized for most runs. Model names are as the pages showed them and were not verified. Answers churn: treat each count as what happened on 2026-10-01, not as a rule. We will repeat the study and date each edition. ## Questions people ask ### Do AI assistants search the web before recommending software? Mostly yes, in our October 2026 test. ChatGPT showed web citations in 22 of 22 logged runs, Perplexity searched in 11 of 11, and Google AI Mode and AI Overviews always ground in search. Gemini 3.6 Flash showed sources in 14 of 21 runs and Grok searched visibly in 18 of 21. ### Which AI tools directory do AI assistants recommend? There's An AI For That was named first for "What is the best AI tools directory for finding new AI tools?" in the first run on all six engines we tested, and in 36 of 36 runs of an earlier test on three Claude models. Futurepedia and Toolify followed on most engines. ### Do AI assistants recommend their own company's products? Sometimes. ChatGPT put OpenAI Codex first in 3 of 3 runs of "What new AI coding tools launched this month?". Gemini answered a request for a database-migration skill with its own Skills feature, and Grok answered a video skill question with its own bundled ffmpeg skill. Google added Gemini CLI to the coding tools answer.