# Googlebot user agent strings: every Google crawler and what it does

> The Googlebot user agent strings for Smartphone and Desktop, the other Google crawlers you will meet in a log, the robots.txt token each obeys, and the Bingbot strings for comparison.

Updated 2026-09-26 · Technical SEO · HTML version: https://getreport.app/guides/googlebot-user-agents

The Googlebot user agent is the string Google's main crawler sends with every request, and it always contains `Googlebot/2.1` and a link to `http://www.google.com/bot.html`. Googlebot Smartphone, which does most of the crawling, looks like Chrome on an Android phone; Googlebot Desktop looks like desktop Chrome. Around them sit a dozen other Google crawlers with their own strings, from Googlebot-Image to AdsBot and GoogleOther. This guide lists them, explains what each one does and which robots.txt rules it follows, and shows how to find each in your server log. For the bigger picture of how Google crawls, see the [guide to Googlebot and crawling](https://getreport.app/guides/googlebot-and-crawling).

## Quick answer

- **Googlebot Smartphone:** Chrome on Android, ending in `(compatible; Googlebot/2.1; +http://www.google.com/bot.html)`. It makes most of Google's requests.
- **Googlebot Desktop:** desktop Chrome with the same `Googlebot/2.1` token, or the short legacy string `Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)`.
- **`W.X.Y.Z` is a placeholder** for the current Chrome version, which changes as Google updates its renderer. Match on the token, never on the full string.
- **One robots.txt token, `Googlebot`, covers both.** You cannot give Smartphone and Desktop different rules.
- **Other Google crawlers carry their own names:** Googlebot-Image, Googlebot-Video, Storebot-Google, GoogleOther, Google-InspectionTool, AdsBot-Google and more.
- **A user agent is a claim, not proof.** Scrapers copy it; [verify the IP address](https://getreport.app/guides/verify-googlebot-fake-googlebot) before you trust or block a visit.

## The two Googlebot user agent strings

Google's documentation publishes the strings with `W.X.Y.Z` where the Chrome version goes. In a real log you will see a version number such as `Chrome/129.0.6668.100` in its place.

**Googlebot Smartphone**

```text
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
```

**Googlebot Desktop**

```text
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36
```

Google also lists two older desktop forms you may still see occasionally:

```text
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.google.com/bot.html)
```

The Nexus 5X and Android 6.0.1 in the smartphone string are fixed; they do not mean Google renders your page on an old phone. Rendering happens in an up-to-date Chromium, and the Chrome version in the string tracks it.

### Why Smartphone dominates

Google indexes the mobile version of pages (mobile-first indexing), so Googlebot Smartphone fetches most HTML pages and its view is the one that gets indexed. Googlebot Desktop still visits, but as a minority. If your log shows mostly Desktop hits, look again: the smartphone string also contains "Chrome" and "Safari", and a filter that matches "Mobile" or "Android" is the easiest way to split them.

## Every Google crawler you will meet in a log

Google groups its crawlers into three kinds, and they behave differently towards robots.txt.

### Common crawlers

These always obey robots.txt.

| Crawler | Appears in the user agent as | robots.txt token | What it fetches |
| --- | --- | --- | --- |
| Googlebot Smartphone / Desktop | `Googlebot/2.1` | `Googlebot` | Pages for Google Search, Discover and other Search features |
| Googlebot Image | `Googlebot-Image/1.0` | `Googlebot-Image` | Images for Google Images and image features |
| Googlebot Video | `Googlebot-Video/1.0` | `Googlebot-Video` | Video files for video features |
| Googlebot News | Uses the Googlebot strings | `Googlebot-News` | News content; there is no separate user agent |
| Google StoreBot | `Storebot-Google/1.0` | `Storebot-Google` | Product, cart and checkout pages for Google Shopping |
| Google-InspectionTool | `Google-InspectionTool/1.0` | Follows `Googlebot` rules | URL Inspection in Search Console and the Rich Results Test |
| GoogleOther | `GoogleOther` (also `GoogleOther-Image`, `GoogleOther-Video`) | `GoogleOther` | One-off crawls by Google product teams, such as research |
| Google-CloudVertexBot | `Google-CloudVertexBot` | `Google-CloudVertexBot` | Crawls requested by site owners building Vertex AI agents |

`Google-Extended` is also a robots.txt token Google documents, but it never appears in a log: it has no user agent. It tells Google whether content its crawlers already fetch may be used for Gemini training and grounding.

If you write no group for `Googlebot-Image` or `Googlebot-News`, they follow your `Googlebot` group. A `Google-InspectionTool` hit in your log is you, or someone with access to your Search Console, testing a URL.

### Special-case crawlers

These do specific jobs for Google products, and some ignore the `User-agent: *` group, so a general `Disallow` does not stop them.

| Crawler | In the user agent | Note |
| --- | --- | --- |
| AdsBot-Google, AdsBot-Google-Mobile | `AdsBot-Google` | Checks the quality of Google Ads landing pages. Ignores `*`; name it to restrict it |
| Mediapartners-Google | `Mediapartners-Google` | Fetches pages that show AdSense ads to match ads to content |
| APIs-Google | `APIs-Google` | Delivers push notifications for Google APIs |
| Google-Safety | `Google-Safety` | Abuse and malware checks; ignores robots.txt |

### User-triggered fetchers

These fetch a page because a person asked for it, so they generally ignore robots.txt. Examples include `FeedFetcher-Google` (RSS and Atom feeds for Google News and others), `Google-Site-Verification` (checking your Search Console verification file or tag), `Google-Read-Aloud`, `Google-NotebookLM` and `Google-Agent`, which Google added in March 2026 for AI agents running on its infrastructure that browse and act on a user's behalf.

Google updates this list several times a year. The authoritative, current version is on Google's crawler documentation pages for common crawlers, special-case crawlers and user-triggered fetchers.

## How to find Googlebot in your log

Your server's access log records the user agent in the last quoted field of each line in the Apache and nginx combined format. If you have not opened one before, [reading Apache and nginx access logs](https://getreport.app/guides/reading-access-logs-apache-nginx) explains where it is and what each field means.

```bash
# All requests claiming to be Googlebot (Search), excluding the Image/Video variants
grep 'Googlebot/2.1' access.log | wc -l

# Split smartphone and desktop
grep 'Googlebot/2.1' access.log | grep -c 'Android'      # smartphone
grep 'Googlebot/2.1' access.log | grep -vc 'Android'     # desktop

# Googlebot's most requested URLs
grep 'Googlebot/2.1' access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

# Every Google crawler by name
grep -oE 'Googlebot[-A-Za-z]*|Storebot-Google|GoogleOther[-A-Za-z]*|Google-[A-Za-z]+|AdsBot-Google[-A-Za-z]*|Mediapartners-Google|FeedFetcher-Google' access.log | sort | uniq -c | sort -rn
```

Or skip the shell:

> **Free tool:** [Log file analysis for SEO: free log analyzer](https://getreport.app/tools/log-analyser): Free log file analysis for SEO in your browser: see what Googlebot and AI crawlers fetch, the 404s they hit and which bots are fake. Nothing is uploaded.

The log file analyser groups every hit by crawler, including Googlebot, Googlebot-Image, Googlebot-Video, Google-InspectionTool, GoogleOther, AdsBot, Mediapartners, Storebot and FeedFetcher, with hits per day, status codes and top URLs. It runs in your browser, and an optional check verifies Googlebot's IP addresses by reverse DNS.

## Writing robots.txt rules for Google's crawlers

Rules are grouped by token, and each crawler follows the most specific group that names it. A few patterns cover most needs:

```text
# Keep images in one folder out of Google Images, allow everything else
User-agent: Googlebot-Image
Disallow: /private-images/

# Keep Google Shopping's crawler out of the cart
User-agent: Storebot-Google
Disallow: /cart/

# Opt out of Gemini training and grounding (no effect on Search)
User-agent: Google-Extended
Disallow: /

User-agent: *
Disallow: /search/
```

Note that a crawler which finds a group naming it ignores the `*` group completely, so the `Storebot-Google` group above lets it crawl `/search/`. Repeat shared rules in each named group. Test the result with the [robots.txt tester](https://getreport.app/tools/robots-txt-tester), which shows the matching rule for each URL and crawler. The report checks the basics too:

> **Check: robots.txt is present.** robots.txt tells crawlers which parts of the site to skip and where the sitemap is. Without one, crawlers use their own defaults and may waste time on search results or admin pages.
>
> 1. Publish a plain-text robots.txt at the site root with at least "User-agent: *", "Allow: /" and a "Sitemap:" line.
> 2. Most CMSs generate one; in WordPress it is under the SEO plugin's Tools section.

> **Check: robots.txt allows this page.** A Disallow rule stops search engines from crawling this page, so they cannot read its content or see any updates. The page can still appear in results as a bare URL with no description.
>
> 1. Remove or narrow the Disallow rule in robots.txt so this URL is allowed for all crawlers.
> 2. If the page should stay out of search, allow crawling and use a noindex tag instead; a blocked page cannot be de-indexed.

## Bingbot user agents for comparison

Bing's crawler sends `bingbot/2.0` in a desktop and a mobile variant, with a Chrome version that changes over time:

```text
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36

Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
```

The older `Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)` still appears too. The robots.txt token is `bingbot`, and unlike Googlebot, Bingbot honours `Crawl-delay`. Bing's other agents include `adidxbot` for Bing Ads and `BingPreview` / `MicrosoftPreview` for page previews.

## Common mistakes

- **Matching the full string.** The Chrome version changes every few weeks; a firewall rule or analytics filter on the exact string silently stops matching. Match `Googlebot` instead.
- **Treating every Googlebot line as Google.** Scrapers send the same string. Verify addresses before drawing conclusions from volumes.
- **Blocking AdsBot with `User-agent: *`.** It ignores that group. If you run Google Ads, you usually want AdsBot to crawl anyway.
- **Expecting a Google-Extended line in the log.** It is only a token; the fetching is done by Googlebot.
- **Serving different content by user agent.** Showing Googlebot something visitors do not see is cloaking and breaks Google's spam policies.

## Questions people ask

### What does Googlebot Smartphone's user agent look like?

It looks like Chrome on an Android phone: `Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)`. W.X.Y.Z is replaced by the current Chrome version. The phone model is fixed and does not describe how your page is rendered.

### Why does the Chrome version in Googlebot's user agent keep changing?

Because Googlebot renders pages with an evergreen Chromium, and the user agent reports the version it currently uses. Google updates it regularly, so a log from last month and one from today show different numbers. Any filter, firewall rule or bot allow-list should match the Googlebot token rather than the full string, or it will break at the next update.

### Can I set different robots.txt rules for Googlebot Smartphone and Desktop?

No. Both crawlers use the same robots.txt token, `Googlebot`, so one group governs both. Google chose this deliberately, because with mobile-first indexing the smartphone crawler's view is the one indexed. If you need to keep content from desktop users or crawlers, that is a page design question, not a robots.txt one.

### What is Google-InspectionTool in my logs?

It is the crawler behind Search Console's URL Inspection tool and the Rich Results Test. A hit from it means someone, usually you or a colleague with Search Console access, tested that URL. It follows your Googlebot rules in robots.txt. Like any user agent it can be faked, so verify the address if the volume looks odd.

### Does Googlebot-Image follow my Googlebot rules?

Yes, when you have not written a group for it. Googlebot-Image uses the most specific group in robots.txt that names it; if there is no `User-agent: Googlebot-Image` group, it falls back to the `Googlebot` group, and then to `*`. Write a dedicated group only if you want images crawled differently from pages.