Most AI crawlers cannot execute JavaScript. OpenAI's GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic's ClaudeBot and PerplexityBot read the HTML your server sends and nothing more; if your text, prices or links appear only after scripts run, those crawlers see a page without them. The main exception is Google, whose AI Overviews, AI Mode and Gemini draw on Googlebot's rendering. This guide lists which crawlers render and which do not, what that hides on a typical site, how to test your own pages and how to fix them, framework by framework. It is part of the guide to generative engine optimization.
Quick answer
- No for the big AI crawlers. A Vercel and MERJ analysis published in December 2024 found that none of the major AI crawlers rendered JavaScript, including OpenAI's, Anthropic's, Meta's, ByteDance's and Perplexity's.
- Some download scripts but do not run them. GPTBot and ClaudeBot fetched JavaScript files, reading them as text.
- Google is the exception. AI Overviews, AI Mode and Gemini use Google's index and its rendering, so Google can see script-built content, with a delay.
- What they miss: client-rendered text, prices loaded by API calls, reviews injected by widgets, meta tags and JSON-LD added by scripts, and links that exist only as click handlers.
- The fix is server-side rendering or static generation for anything you want read and cited. Keep JavaScript for interaction.
- Test it with the free JavaScript rendering checker, which compares your raw HTML with the rendered page.
Which AI crawlers render JavaScript?
The most detailed public data is the Vercel and MERJ study "The rise of the AI crawler" (December 2024), which analysed AI crawler traffic across Vercel's network. Its core finding: the major AI crawlers fetch raw HTML and extract text from it. OpenAI's and Anthropic's crawlers did request JavaScript files, GPTBot in 11.50% and ClaudeBot in 23.84% of their fetches, but did not execute them.
| Crawler | Operator | Purpose | Runs JavaScript? |
|---|---|---|---|
| Googlebot (feeds AI Overviews, AI Mode, Gemini grounding) | Search and AI features | Yes, in an evergreen Chromium, sometimes with a delay | |
| Applebot | Apple | Search, Siri, Spotlight | Yes, according to the Vercel and MERJ study |
| GPTBot | OpenAI | Training | No |
| OAI-SearchBot | OpenAI | ChatGPT search | No |
| ChatGPT-User | OpenAI | Fetch for a user | No |
| ClaudeBot | Anthropic | Training | No |
| PerplexityBot | Perplexity | AI search | No |
| Meta-ExternalAgent | Meta | Training | No |
| Bytespider | ByteDance | Training | No |
| CCBot | Common Crawl | Open web archive used for training | No; it stores the raw responses |
Anthropic's Claude-SearchBot and Claude-User were not part of that study. Nothing Anthropic has published suggests they render, so plan as if they do not.
Two caveats. First, the study is a snapshot from late 2024; operators could add rendering later, and none has announced it as of September 2026. Second, browser agents are different: tools that operate a browser for a user, such as ChatGPT's agent mode or Google's Google-Agent, can load a page in a real browser and run its scripts. They visit one page for one person, though. They do not build the indexes AI answers are drawn from.
Why they do not render
Rendering means loading each page in a headless browser, running its scripts and waiting for the result. At web scale that costs far more computing time than downloading HTML, and Google built a dedicated system, its Web Rendering Service, to do it. AI crawlers that fetch billions of pages mostly skip that step.
What AI crawlers miss on a typical site
A page can look complete in your browser and be half-empty to a crawler. The usual gaps:
- Client-rendered apps. A React, Vue or Angular app without server rendering sends
<div id="root"></div>and a script tag. The crawler sees no text at all. - Data loaded after the page. Prices, stock levels, specifications or articles fetched from an API once the page has loaded.
- Third-party widgets. Reviews, FAQs, store locators and comparison tables injected by an embed script.
- Head tags set by scripts. A title, description, canonical or robots tag changed by JavaScript. The crawler reads the version in the HTML.
- Structured data added by a tag manager. JSON-LD inserted through Google Tag Manager or a plugin's front-end script exists only after rendering. Google may see it; other crawlers do not.
- Links that are not links. Navigation built with click handlers instead of
<a href>, so crawlers cannot follow it to the rest of the site. - Infinite scroll and "load more" without paginated URLs.
Content that is in the HTML but hidden until a click, such as tabs, accordions and "read more" sections, is fine. The crawler reads the HTML; it does not care what is visible.
How to see what an AI crawler sees
In your browser
View Source (Ctrl+U or Cmd+Option+U) shows the raw HTML, which is what AI crawlers read. The Elements panel in developer tools shows the rendered page. Search View Source for a sentence from your main content or a price. If it is not there, AI crawlers do not see it.
From the command line
# Fetch the page as an AI crawler would: raw HTML, no scripts
curl -s -A "Mozilla/5.0 (compatible; GPTBot/1.3; +https://openai.com/gptbot)" https://example.com/products/blue-mug \
| grep -c "Hand-glazed stoneware"A count of 0 means the sentence is not in the server's HTML. Sending a crawler's user agent also shows whether your server or CDN answers bots differently, for example with a challenge page. The version number in that user agent is only an example; OpenAI documents the current strings on its crawler page.
With getReport
The JavaScript rendering checker loads the page twice, as the server sends it and after Chromium has run every script, and lists what differs field by field: title, canonical, robots tag, headings, word count, links and JSON-LD. Tags that JavaScript changes, a canonical or noindex above all, are flagged first. Every getReport report runs the same comparison:
The AI crawler check combines the same comparison with what your robots.txt tells 18 AI crawlers, so you see access and readability together.
How to fix it
The principle is the same whatever the stack: the server sends HTML that already contains the content, and JavaScript adds interactivity on top. That is server-side rendering (SSR), static generation (SSG) or a mix.
| Stack | What to do |
|---|---|
| Next.js | App Router renders on the server by default. Keep content in server components; avoid fetching main content in useEffect |
| Nuxt | Server rendering is on by default; do not switch to ssr: false for content pages |
| SvelteKit, Remix, Astro | Server-rendered or static by default; keep it that way for content routes |
| Angular | Add server rendering with @angular/ssr |
| React or Vue single-page app (Vite, Create React App) | Move content routes to a framework with SSR or SSG, or prerender them at build time |
| WordPress, Drupal, Shopify | Themes render on the server; check plugins and apps that inject content, such as reviews and FAQs |
For a deeper walk-through, SSR vs CSR for SEO compares the rendering modes, and the JavaScript SEO guide covers the Google side.
Widgets and tag-managed JSON-LD
For third-party reviews, check whether the provider offers a server-side or static option that writes the reviews into your HTML. For structured data, output JSON-LD from your templates rather than from a tag manager, so it is in the raw HTML:
Prerendering as a stopgap
If you cannot change the stack soon, a prerendering service or build step can serve rendered HTML snapshots. Serve the same content to crawlers and people; showing bots something different is cloaking. Google describes dynamic rendering, serving bots a different rendering path, as a workaround rather than a long-term solution.
How to verify the fix
- View Source on the fixed page: your main text, prices and links should be there.
- Run the rendering checker again: the "only after JavaScript" word count should be near zero.
- A few weeks later, check your server log for OAI-SearchBot, ClaudeBot and PerplexityBot requests to the fixed pages returning 200 with a realistic response size. A 2 KB response for a product page is a shell.
Common mistakes
- Testing in your own browser only. It runs JavaScript, so everything looks fine.
- Assuming Google's rendering covers AI search. It covers Google's features only.
- Fixing the body and forgetting the head. A title, canonical or JSON-LD set by scripts is still missing from the raw HTML.
- Blocking crawlers from JavaScript and CSS to save bandwidth. It does not help AI crawlers and stops Googlebot rendering the page.
- Prerendering a different page for bots. Keep the content identical, or it becomes cloaking.
Questions people ask
Does ChatGPT see content loaded by JavaScript?
No. OpenAI's crawlers, GPTBot, OAI-SearchBot and ChatGPT-User, read the HTML your server returns and do not run scripts, according to the Vercel and MERJ analysis of AI crawler traffic. Text, prices or reviews that appear only after JavaScript runs are invisible to ChatGPT search. Render that content on the server or at build time so it is in the HTML.
Does Perplexity render JavaScript?
No. PerplexityBot was among the AI crawlers the Vercel and MERJ study found not rendering JavaScript: it reads the raw HTML. Perplexity-User, which fetches a page when someone asks about it, has not been shown to render either. If your pages are built in the browser, Perplexity sees little or no content. Server-side rendering or static generation fixes it.
Can AI crawlers read JSON-LD added by Google Tag Manager?
Mostly not. Tag Manager inserts JSON-LD with JavaScript after the page loads, so it exists only in the rendered page. Google can read it after rendering, but AI crawlers that do not run scripts see only the raw HTML, where it is missing. Output structured data from your page templates instead, so it is in the HTML every crawler receives.
Do Google AI Overviews see JavaScript-rendered content?
Yes, in principle. AI Overviews and AI Mode are built from Google's search index, and Googlebot renders pages in Chromium before indexing, so content added by scripts can be used. Rendering can be delayed or incomplete, though, and other AI products do not render at all, so important content is still safest in the server's HTML.
Is serving prerendered pages to AI crawlers cloaking?
Not if the content matches what people see. Serving a prerendered HTML snapshot of the same page is a legitimate way to make content readable without JavaScript. It becomes cloaking when bots get different text, links or offers from visitors. Google calls dynamic rendering a workaround, so treat prerendering as a bridge to real server-side rendering.