# Shopify robots.txt: how to edit robots.txt.liquid safely

> Shopify generates robots.txt for every store and lets you change it through a robots.txt.liquid theme template. What the defaults block, how to add, remove and extend rules safely, and how to undo a mistake.

Updated 2026-09-26 · Other platforms · HTML version: https://getreport.app/guides/shopify-robots-txt

Every Shopify store has a robots.txt that Shopify generates for you, with rules that keep crawlers out of the cart, checkout, account pages, internal search and sorted or tag-filtered collection URLs. You cannot upload your own file, but you can change it by adding a `robots.txt.liquid` template to your theme, which outputs Shopify's default rules through a Liquid loop and lets you add, remove or extend them. This guide shows what the Shopify robots.txt blocks by default, how to create and edit the template without losing those defaults, the changes that are worth making, and how to go back if something breaks. For robots.txt in general, start with the [robots.txt guide](https://getreport.app/guides/robots-txt).

## Quick answer

- **See the current file** at `https://yourstore.com/robots.txt`.
- **The defaults are good.** Most stores never need to change them. Shopify's own documentation recommends keeping the default rules.
- **To edit:** Online Store → Themes → the three-dot menu on your live theme → Edit code → Templates → Add a new template → robots.txt.
- **Keep the Liquid loop** that prints `robots.default_groups`, and add your rules inside or after it. Never replace it with a hard-coded copy.
- **To reset:** delete `robots.txt.liquid`. Shopify goes back to the default file, but the deleted template cannot be recovered, so save a copy first.
- **Test after every change** with the [robots.txt tester](https://getreport.app/tools/robots-txt-tester): a product, a collection and the home page must stay allowed.

## What Shopify's default robots.txt blocks

Open `/robots.txt` on your store. The file starts with a comment that the store uses Shopify, then a `User-agent: *` group with rules for, among others:

| Blocked path pattern | Why |
| --- | --- |
| `/admin`, `/cart`, `/checkout`, `/checkouts/`, `/orders`, `/account` | Private and transactional pages with no search value |
| `/collections/*sort_by*` | The same collection in a different sort order |
| `/collections/*+*` and encoded forms such as `*%2B*` | Collections filtered by several tags at once |
| `/blogs/*+*` | Blog listings filtered by several tags |
| `/search` | Internal search results |
| `/policies/` | Policy pages, which are duplicated across stores |
| Preview and internal paths, such as `*preview_theme_id*` | Theme previews and system URLs |

The same rules repeat for market or language prefixes, such as `/*/collections/*sort_by*`, so translated and regional URLs are covered too. Then comes the `Sitemap:` line for the store's `sitemap.xml`, followed by separate groups for a few named crawlers, such as `adsbot-google`, which ignores `*` groups and needs its own rules, and `Nutch`, which is blocked completely.

Shopify updates these defaults over time, so treat your live file as the reference, not any list you find online.

## How to create robots.txt.liquid

1. In the Shopify admin, go to **Online Store → Themes**.
2. On your live theme, open the **…** menu and choose **Edit code**.
3. In the **Templates** folder, click **Add a new template**.
4. Choose **robots.txt** and create it. Shopify creates `templates/robots.txt.liquid` with the default code.

The template belongs to the theme. If you publish a different theme later, copy the template into it, or the store goes back to the default file.

Shopify's help centre calls this an unsupported customization: its support team cannot help you with edits to the file, and it warns that a wrong rule can cost you all your search traffic. Keep a copy of the working template somewhere outside the theme before you change it.

## The default template and the Liquid objects

The template Shopify documents, `templates/robots.txt.liquid`, looks like this:

```liquid
{% for group in robots.default_groups %}
  {{- group.user_agent }}

  {%- for rule in group.rules -%}
    {{ rule }}
  {%- endfor -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}
```

It prints Shopify's defaults group by group:

- **`robots.default_groups`** is the list of default groups.
- **`group.user_agent`** prints the `User-agent:` line; `group.user_agent.value` is the name alone, such as `*`.
- **`group.rules`** holds the `Allow` and `Disallow` rules; each `rule` has a `directive` and a `value`.
- **`group.sitemap`** prints the `Sitemap:` line where a group has one.

Because the loop reads Shopify's current defaults every time, your store keeps receiving Shopify's updates to them. That is the reason never to paste the output of `/robots.txt` into the template as plain text: it freezes the rules on the day you copied them.

## How to edit the Shopify robots.txt safely

### Add a rule to the `*` group

Add your rule inside the loop, after the default rules, for the group you want. This example from Shopify's documentation blocks internal search URLs with a `q` parameter for all crawlers:

```liquid
{% for group in robots.default_groups %}
  {{- group.user_agent }}

  {%- for rule in group.rules -%}
    {{ rule }}
  {%- endfor -%}

  {%- if group.user_agent.value == '*' -%}
    {{ 'Disallow: /*?q=*' }}
  {%- endif -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}
```

Rules added this way sit next to Shopify's, in the same group, so they apply to every crawler that uses the `*` group.

### Remove a default rule

Wrap the rule output in an `unless` that skips the rule you want gone. This removes the `/policies/` block:

```liquid
  {%- for rule in group.rules -%}
    {%- unless rule.directive == 'Disallow' and rule.value == '/policies/' -%}
      {{ rule }}
    {%- endunless -%}
  {%- endfor -%}
```

Remove defaults only with a reason. Unblocking sort and tag-filter URLs, for example, lets crawlers into thousands of near-duplicate collection pages.

### Add a group for another crawler

Groups for other user agents go after the loop, as plain text:

```liquid
{% comment %} after the endfor of the default loop {% endcomment %}

User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /
```

A crawler that finds a group with its own name follows only that group and ignores `*`. So a `User-agent: Googlebot` group here would make Googlebot ignore all of Shopify's default rules, which you almost never want. Use named groups only to block a crawler completely, as above. [AI crawlers and robots.txt](https://getreport.app/guides/ai-crawlers-and-robots-txt) lists the tokens for training and answer crawlers and the trade-offs of blocking each.

### Add an extra sitemap

If an app or a second storefront has its own sitemap, add a line after the loop with the full URL:

```liquid
Sitemap: https://yourstore.com/pages/extra-sitemap.xml
```

## Changes worth making, and ones to avoid

Worth considering:

- **Blocking AI training crawlers,** if that is your policy.
- **Blocking app-generated URL patterns** that create endless variations, such as some filter or search apps. Check a few of those URLs first; if they carry a canonical to the main collection, a block may not be needed.
- **Blocking internal search variants** that the defaults miss, as in Shopify's example above.

Avoid:

- **Blocking `/products/` or `/collections/` paths** to hide a few items. Use a `noindex` in the theme for those pages instead, so Google can see it.
- **Blocking `/cdn/` or asset paths.** Google needs your CSS, JavaScript and images to render the page.
- **Replacing the loop with a static copy** of the file.
- **A `Googlebot` group with a handful of rules,** which silently cancels every default rule for Google.

Filtered collection pages are a judgment call that depends on how your theme and filters build URLs; the [faceted navigation guide](https://getreport.app/guides/faceted-navigation-and-parameter-urls) helps you decide between blocking, canonicalising and leaving them alone. [robots.txt patterns](https://getreport.app/guides/robots-txt-patterns-wildcards-allow-crawl-delay) explains how `*`, `$` and the longest-match rule decide what a line blocks.

## How to check your Shopify robots.txt

Save the template, then open `/robots.txt` in a private window. The change is live immediately. Look for obvious problems: missing line breaks between rules, a Liquid error, or your new lines landing in the wrong group.

> **Free tool:** [robots.txt tester: check and validate robots.txt](https://getreport.app/tools/robots-txt-tester): Free robots.txt tester: see whether robots.txt blocks a page for Googlebot, whether your sitemap is declared and reachable, and if the page can be indexed.

Enter a product URL, a collection URL and the home page. The tester fetches the live file, parses it the way Google does and says whether each URL is allowed for Googlebot and for all other agents, naming the rule if one blocks it. It also fetches the sitemap the file declares and reads its URL count, and checks the page for `noindex`. Then test a URL you meant to block, such as a sorted collection, to confirm the rule matches.

> **Check: robots.txt allows this page.** A Disallow rule stops search engines from crawling this page, so they cannot read its content or see any updates. The page can still appear in results as a bare URL with no description.
>
> 1. Remove or narrow the Disallow rule in robots.txt so this URL is allowed for all crawlers.
> 2. If the page should stay out of search, allow crawling and use a noindex tag instead; a blocked page cannot be de-indexed.

> **Check: robots.txt declares a sitemap.** A "Sitemap:" line in robots.txt is how crawlers find your XML sitemap without you registering it anywhere. Without it, new pages are discovered later.
>
> 1. Add a line to robots.txt: Sitemap: https://your-site.com/sitemap.xml (use the full URL).

Google picks up the new file within about 24 hours. In Search Console, Settings → robots.txt shows the version Google has and lets you request a fresh fetch. For the rest of what you can and cannot change on the platform, see [what you can and cannot fix on Shopify](https://getreport.app/guides/shopify-what-you-can-and-cannot-fix).

## How to undo a mistake

If the file is broken or you are unsure what changed, delete `templates/robots.txt.liquid` in the code editor. Shopify then serves its default robots.txt again. Deleted templates cannot be restored, which is why you keep a copy of your customised version before editing; paste it into a new template when you are ready.

## Common mistakes

- **Pasting the live file into the template** instead of keeping the loop. The rules stop receiving Shopify's updates.
- **Editing the template in an unpublished theme** and expecting the live file to change.
- **Publishing a new theme** without copying `robots.txt.liquid` across.
- **Missing line breaks.** Liquid whitespace control (`{%-` and `-%}`) can glue two rules onto one line. Check the output.
- **Adding `Noindex:` lines.** Google ignores them; see [why noindex in robots.txt does not work](https://getreport.app/guides/robots-txt-noindex).
- **Blocking products to hide them.** They can stay in Google as bare URLs.

## Questions people ask

### Can you edit robots.txt on Shopify?

Yes, since 2021. Add a `robots.txt.liquid` template in your theme's code editor: Online Store → Themes → Edit code → Templates → Add a new template → robots.txt. The template prints Shopify's default rules through a Liquid loop, and you add, remove or extend rules around it. You cannot upload a plain robots.txt file.

### Why does Shopify block collections in robots.txt?

It blocks only variations of collection pages, not the collections themselves. The default rules stop crawlers from fetching collections sorted in a different order (`sort_by`) or filtered by several tags at once (`+` in the URL), which would otherwise create thousands of near-duplicate pages. The main collection URLs stay open to crawlers.

### How do I reset my Shopify robots.txt to the default?

Delete the `robots.txt.liquid` template from your live theme in the code editor. Shopify then serves its default robots.txt again. A deleted template cannot be recovered, so copy your customised version somewhere safe first if you might want it back.

### Should I block AI bots in my Shopify robots.txt?

It depends on your goal. Blocking training crawlers such as GPTBot and CCBot keeps your product pages out of future model training; blocking answer crawlers such as OAI-SearchBot or PerplexityBot keeps you out of AI answers that could link to your store. Add a named group with `Disallow: /` after the default loop for each crawler you decide to block.