AI search engines are search products that write an answer instead of, or on top of, a list of links: ChatGPT search, Perplexity, Google's AI Overviews and AI Mode, Microsoft Copilot and Claude. For a site owner they are not interchangeable; each answers from a different index, crawls with different user agents, cites differently, and gives you different switches. This page is the reference: who feeds whom, which crawler names belong to which product, and the one thing to do for each. The work of actually optimising for them is covered in our pillar guide to generative engine optimization; come back here whenever you need to know which engine a crawler name or a referrer belongs to.
Quick answer
- Five engines matter in 2026: ChatGPT search, Google's AI Overviews and AI Mode, Microsoft Copilot, Perplexity and Claude. Together they cover almost all AI answer traffic to ordinary sites.
- Two answer from classic indexes: Google's AI features use Google's Search index, and Copilot grounds answers in Bing. Being indexed there is most of the work.
- Three run their own crawlers for search: OpenAI's OAI-SearchBot, Perplexity's PerplexityBot and Anthropic's Claude-SearchBot. Blocking one of them removes you from that product's answers, not the others.
- Training and search are separate switches everywhere: GPTBot, ClaudeBot and Google-Extended govern model training, not citations.
- Only Google and Bing render JavaScript. The Vercel and MERJ analysis from December 2024 found no major AI crawler executing scripts, so text that needs JavaScript is invisible to the other three.
- Check all of it in a minute with the free AI crawler check.
The AI search engines compared
The table is the whole landscape in one view. Every crawler name and policy in it comes from the operator's own documentation; the sections below add the detail per engine.
| Engine | Answers from | Crawlers to know | How it cites | Renders JavaScript | Your switch |
|---|---|---|---|---|---|
| ChatGPT search | OpenAI's own crawl plus third-party search providers (Bing has been its announced partner since 2023) | OAI-SearchBot (search), ChatGPT-User (user fetch), GPTBot (training) | Inline citations and a source list | No | robots.txt per bot; blocking OAI-SearchBot removes you from answers in about 24 hours |
| Google AI Overviews + AI Mode | Google's Search index | Googlebot; Google-Extended is a training token, not a crawler | "Supporting links" next to the text they support | Yes, via Googlebot | Search generative AI control in Search Console; nosnippet, data-nosnippet, max-snippet |
| Microsoft Copilot | Bing's index | Bingbot | Linked sources; Bing Webmaster Tools counts the citations | Yes, Bingbot renders | Bing indexing; nocache and noarchive meta tags limit use in answers |
| Perplexity | Its own index, built by its own crawler | PerplexityBot (search), Perplexity-User (user fetch) | Numbered citations on every answer | No | robots.txt for PerplexityBot; Perplexity-User generally ignores it |
| Claude | Anthropic's own Claude-SearchBot index; Brave Search is listed on Anthropic's subprocessor page as its web search vendor | Claude-SearchBot (search), Claude-User (user fetch), ClaudeBot (training) | Citations shown with answers | No | robots.txt per bot; blocking ClaudeBot does not affect the other two |
Two patterns are worth noticing before the details. First, every operator separates training from search: you can keep your content out of model training and still be cited, as long as you block by purpose rather than by company. Second, the JavaScript column splits the field in half. Google and Microsoft bring twenty years of rendering infrastructure; the newer crawlers read only the HTML your server sends, a result the Vercel and MERJ crawler-traffic analysis documented in December 2024 and no operator has contradicted since.
ChatGPT search
ChatGPT is the biggest AI assistant by a wide margin (OpenAI said in February 2026 that it had passed 900 million weekly users) and when a question needs current information it searches the web, reads a handful of pages and writes an answer with inline citations. Its sources come from two directions: OpenAI's own crawler, OAI-SearchBot, and third-party search providers. OpenAI does not name its current providers; Microsoft and OpenAI announced Bing as the partner in 2023, which is why Bing indexing keeps showing up in ChatGPT advice.
Three OpenAI user agents matter, and they do different jobs. OAI-SearchBot builds the search side; OpenAI documents that sites which disallow it are not shown in ChatGPT search answers. ChatGPT-User fetches a page live when a user's question needs it. GPTBot collects training data and has no effect on search: blocking it is a separate decision.
The one thing to do: allow OAI-SearchBot in robots.txt and at your firewall, and verify your site in Bing Webmaster Tools. The full walk-through, including product feeds for shops and how to measure with the utm_source=chatgpt.com tag, is in our guide to SEO for ChatGPT.
Google AI Overviews and AI Mode
Google's AI features are the largest AI search surface, because they sit inside Google Search itself. AI Overviews are the AI-written summaries above the results; AI Mode is the fully conversational version, which Google made the default search experience at I/O in May 2026. Both are built the same way: pages are retrieved from Google's normal Search index by its core ranking systems, often through "query fan-out" (several related searches run for one question) and some are shown as supporting links.
That makes Google the engine with the clearest rules and the least extra work. Google's May 2026 guide to its generative AI features says a page needs nothing beyond being indexed and eligible to show a snippet: no schema requirement, no llms.txt, no special writing style. The common confusion is Google-Extended, a robots.txt token that governs Gemini training and grounding and does nothing to AI Overviews or AI Mode. If you want out, the Search generative AI control in Search Console removes the whole site; the snippet directives limit what can be quoted page by page.
The one thing to do: confirm your important pages are indexed and snippet-eligible, then answer the specific sub-questions your topic fans out into. How to rank in AI Overviews covers eligibility, the controls and what makes a page more likely to be linked.
Microsoft Copilot
Copilot is Microsoft's assistant across Windows, Edge, Office and copilot.microsoft.com, and for web questions it grounds its answers in Bing's index. There is no separate Copilot crawler: Bingbot's crawl is the input, and Bingbot has rendered JavaScript with an evergreen browser engine since 2019, so Copilot's index sees more of a script-heavy site than OpenAI's, Perplexity's or Anthropic's crawlers do.
Copilot is also the engine with the best publisher reporting. Bing Webmaster Tools' AI Performance report counts how often Copilot and Bing's AI answers cited each of your pages and lists the generated "grounding queries" that found them: the only per-page citation count any AI operator provides. For control, Microsoft documented two meta tags in September 2023: nocache limits an answer to your URL, title and snippet, and noarchive keeps the page out of AI answers entirely, both without touching your classic Bing rankings.
The one thing to do: verify the site in Bing Webmaster Tools and submit your sitemap. Most sites never have, and it is ten minutes that buys you both the index Copilot answers from and the only citation report in the business.
Perplexity
Perplexity is the only major product that started as an AI search engine rather than adding search to an assistant. Every answer carries numbered citations, and the company has said its index is its own: PerplexityBot crawls the web to build it, and Perplexity states that PerplexityBot is not used to train foundation models. The last officially disclosed volume was 780 million queries in May 2025, a figure its CEO gave at the Bloomberg Tech conference; the company has not published a comparable number since, so treat bigger claims as estimates.
Two user agents matter. PerplexityBot builds the index and, Perplexity says, respects robots.txt. Perplexity-User fetches a page live when a user asks about it, and the documentation says it generally ignores robots.txt because a person, not a crawler schedule, triggered the request. In practice the common problem is not robots.txt but the firewall: Cloudflare and several security vendors have fought publicly with Perplexity over its fetching, and many default bot rules block it silently.
The one thing to do: check that PerplexityBot is allowed in robots.txt and actually getting 200 responses through your CDN or firewall; a site can say "allowed" and still never be fetched. Your server log, not your analytics, is where the evidence lives.
Claude
Claude added web search in 2025 and answers with citations. Its sources come from two places: Anthropic's own crawler, Claude-SearchBot, which Anthropic documents as improving search result quality, and Brave Search, which Anthropic's subprocessor list names as its web search vendor. Anthropic clarified its three crawlers' roles in a documentation update in February 2026: ClaudeBot collects training data, Claude-SearchBot feeds search, Claude-User fetches a page when a user asks, and blocking one does not block the others.
Claude sends less referral traffic than ChatGPT or Perplexity for most sites, but the cost of being present is near zero because the work overlaps completely: readable HTML, clear answers, and the right robots.txt groups. A blanket "block everything from Anthropic" rule, common in 2023-era robots.txt files, removes you from Claude's citations along with its training set; usually not what the site owner meant.
The one thing to do: treat ClaudeBot and Claude-SearchBot as the separate switches they are. If you block training, add the explicit group for ClaudeBot only, and leave Claude-SearchBot and Claude-User alone.
Whose index answers you: Google's, Bing's, or their own?
The question behind most AI search confusion is where the pages in an answer come from, and the honest answer is: it depends on the engine, and it is converging on "their own, plus borrowed".
- Borrowed entirely: Google's AI features and Copilot answer from the Google and Bing indexes. No new crawler, no new work; classic indexing covers them.
- Own index: Perplexity built its own from the start. Anthropic runs Claude-SearchBot alongside Brave's index.
- Hybrid: ChatGPT search combines OpenAI's own crawling with unnamed third-party providers.
The practical consequence is cheap: rank in Google and Bing, and you are retrievable by every engine that borrows; allow the three operator crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot) and you are retrievable by the ones that do not. There is no engine on the list that requires more than that to consider you.
How visitors arrive from AI search engines
Each engine leaves its own trace: referrers such as chatgpt.com and perplexity.ai in your analytics, user-fetch agents such as ChatGPT-User in your server log, impression counts in Search Console's Generative AI report and citation counts in Bing Webmaster Tools. What each number means, what it misses and how to build a monthly baseline is the subject of our guide to what AI visibility is and how to measure it; this page stays the map of who is who.
Check your access in a minute
The check reads what your site tells the crawlers in the table above (your robots.txt verdict for 18 AI user agents from OpenAI, Anthropic, Google, Perplexity and others, grouped by purpose) validates your llms.txt, looks for noai and TDM opt-out tags, and measures how much of your text only exists after JavaScript runs, which three of the five engines never do. The same findings appear in every getReport report:
One run answers the access question for every engine at once, which is why it is the first check to do, whichever engine you care about most.
Questions people ask
What are AI search engines?
AI search engines are products that answer a question with written text and cited sources instead of a ranked list of links: ChatGPT search, Perplexity, Google's AI Overviews and AI Mode, Microsoft Copilot and Claude. Under the answer they still work like search; a crawler builds an index, the question retrieves pages, and a language model writes the answer from what was retrieved, citing some of the pages.
Which AI search engine is the most used?
ChatGPT, by a wide margin: OpenAI said in February 2026 it had passed 900 million weekly users. But Google's AI Overviews and AI Mode reach more people in absolute terms, because they appear inside ordinary Google searches. For a site owner the ranking matters less than coverage, since the same few fixes serve every engine.
Do AI search engines have their own index?
Some do. Perplexity crawls and indexes the web itself with PerplexityBot, and Anthropic runs Claude-SearchBot for Claude's search. Google's AI Overviews and AI Mode use Google's normal Search index, Copilot answers from Bing's, and ChatGPT search mixes OpenAI's own crawling with third-party search providers. So classic Google and Bing indexing covers part of the field, and allowing the operators' own search crawlers covers the rest.
How do I show up in AI search engines?
Four steps cover all five engines: allow the AI search crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot) in robots.txt and at your firewall; be indexed in Google and Bing, which feed AI Overviews, AI Mode and Copilot; serve your main text in plain HTML, because only Google's and Bing's crawlers render JavaScript; and answer questions directly enough to be quoted. Our generative engine optimization guide walks through each step.
Can I block AI search engines?
Yes, and each one separately. Disallow OAI-SearchBot, PerplexityBot or Claude-SearchBot in robots.txt to leave that product's answers; use the Search generative AI control in Search Console to leave Google's AI features without losing rankings; use Microsoft's noarchive tag to stay out of Copilot answers while staying in Bing. User-triggered fetchers such as ChatGPT-User and Perplexity-User may still fetch a page a user explicitly asks about.
What is the best AI search engine?
For searching, pick by need: Perplexity shows its sources most prominently, ChatGPT has the largest feature set, and Google's AI Mode is the default for most people already. For a site owner the question is different; the best engine is whichever one your audience uses, and you rarely have to choose, because access, readable HTML and quotable answers serve all five at once.