Shopify robots.txt: how to edit robots.txt.liquid safely

Every Shopify store has a robots.txt that Shopify generates for you, with rules that keep crawlers out of the cart, checkout, account pages, internal search and sorted or tag-filtered collection URLs. You cannot upload your own file, but you can change it by adding a robots.txt.liquid template to your theme, which outputs Shopify's default rules through a Liquid loop and lets you add, remove or extend them. This guide shows what the Shopify robots.txt blocks by default, how to create and edit the template without losing those defaults, the changes that are worth making, and how to go back if something breaks. For robots.txt in general, start with the robots.txt guide.

Quick answer

• See the current file at https://yourstore.com/robots.txt.
• The defaults are good. Most stores never need to change them. Shopify's own documentation recommends keeping the default rules.
• To edit: Online Store → Themes → the three-dot menu on your live theme → Edit code → Templates → Add a new template → robots.txt.
• Keep the Liquid loop that prints robots.default_groups, and add your rules inside or after it. Never replace it with a hard-coded copy.
• To reset: delete robots.txt.liquid. Shopify goes back to the default file, but the deleted template cannot be recovered, so save a copy first.
• Test after every change with the robots.txt tester: a product, a collection and the home page must stay allowed.

What Shopify's default robots.txt blocks

Open /robots.txt on your store. The file starts with a comment that the store uses Shopify, then a User-agent: * group with rules for, among others:

The same rules repeat for market or language prefixes, such as /*/collections/*sort_by*, so translated and regional URLs are covered too. Then comes the Sitemap: line for the store's sitemap.xml, followed by separate groups for a few named crawlers, such as adsbot-google, which ignores * groups and needs its own rules, and Nutch, which is blocked completely.

Shopify updates these defaults over time, so treat your live file as the reference, not any list you find online.

How to create robots.txt.liquid

1. In the Shopify admin, go to Online Store → Themes.
2. On your live theme, open the … menu and choose Edit code.
3. In the Templates folder, click Add a new template.
4. Choose robots.txt and create it. Shopify creates templates/robots.txt.liquid with the default code.

The template belongs to the theme. If you publish a different theme later, copy the template into it, or the store goes back to the default file.

Shopify's help centre calls this an unsupported customization: its support team cannot help you with edits to the file, and it warns that a wrong rule can cost you all your search traffic. Keep a copy of the working template somewhere outside the theme before you change it.

The default template and the Liquid objects

The template Shopify documents, templates/robots.txt.liquid, looks like this:

{% for group in robots.default_groups %}
  {{- group.user_agent }}

  {%- for rule in group.rules -%}
    {{ rule }}
  {%- endfor -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

It prints Shopify's defaults group by group:

• robots.default_groups is the list of default groups.
• group.user_agent prints the User-agent: line; group.user_agent.value is the name alone, such as *.
• group.rules holds the Allow and Disallow rules; each rule has a directive and a value.
• group.sitemap prints the Sitemap: line where a group has one.

Because the loop reads Shopify's current defaults every time, your store keeps receiving Shopify's updates to them. That is the reason never to paste the output of /robots.txt into the template as plain text: it freezes the rules on the day you copied them.

How to edit the Shopify robots.txt safely

Add a rule to the * group

Add your rule inside the loop, after the default rules, for the group you want. This example from Shopify's documentation blocks internal search URLs with a q parameter for all crawlers:

{% for group in robots.default_groups %}
  {{- group.user_agent }}

  {%- for rule in group.rules -%}
    {{ rule }}
  {%- endfor -%}

  {%- if group.user_agent.value == '*' -%}
    {{ 'Disallow: /*?q=*' }}
  {%- endif -%}

  {%- if group.sitemap != blank -%}
    {{ group.sitemap }}
  {%- endif -%}
{% endfor %}

Rules added this way sit next to Shopify's, in the same group, so they apply to every crawler that uses the * group.

Remove a default rule

Wrap the rule output in an unless that skips the rule you want gone. This removes the /policies/ block:

  {%- for rule in group.rules -%}
    {%- unless rule.directive == 'Disallow' and rule.value == '/policies/' -%}
      {{ rule }}
    {%- endunless -%}
  {%- endfor -%}

Remove defaults only with a reason. Unblocking sort and tag-filter URLs, for example, lets crawlers into thousands of near-duplicate collection pages.

Add a group for another crawler

Groups for other user agents go after the loop, as plain text:

{% comment %} after the endfor of the default loop {% endcomment %}

User-agent: GPTBot
Disallow: /

User-agent: CCBot
Disallow: /

A crawler that finds a group with its own name follows only that group and ignores *. So a User-agent: Googlebot group here would make Googlebot ignore all of Shopify's default rules, which you almost never want. Use named groups only to block a crawler completely, as above. AI crawlers and robots.txt lists the tokens for training and answer crawlers and the trade-offs of blocking each.

Add an extra sitemap

If an app or a second storefront has its own sitemap, add a line after the loop with the full URL:

Sitemap: https://yourstore.com/pages/extra-sitemap.xml

Changes worth making, and ones to avoid

Worth considering:

• Blocking AI training crawlers, if that is your policy.
• Blocking app-generated URL patterns that create endless variations, such as some filter or search apps. Check a few of those URLs first; if they carry a canonical to the main collection, a block may not be needed.
• Blocking internal search variants that the defaults miss, as in Shopify's example above.

Avoid:

• Blocking /products/ or /collections/ paths to hide a few items. Use a noindex in the theme for those pages instead, so Google can see it.
• Blocking /cdn/ or asset paths. Google needs your CSS, JavaScript and images to render the page.
• Replacing the loop with a static copy of the file.
• A Googlebot group with a handful of rules, which silently cancels every default rule for Google.

Filtered collection pages are a judgment call that depends on how your theme and filters build URLs; the faceted navigation guide helps you decide between blocking, canonicalising and leaving them alone. robots.txt patterns explains how *, $ and the longest-match rule decide what a line blocks.

How to check your Shopify robots.txt

Save the template, then open /robots.txt in a private window. The change is live immediately. Look for obvious problems: missing line breaks between rules, a Liquid error, or your new lines landing in the wrong group.

Enter a product URL, a collection URL and the home page. The tester fetches the live file, parses it the way Google does and says whether each URL is allowed for Googlebot and for all other agents, naming the rule if one blocks it. It also fetches the sitemap the file declares and reads its URL count, and checks the page for noindex. Then test a URL you meant to block, such as a sorted collection, to confirm the rule matches.

Google picks up the new file within about 24 hours. In Search Console, Settings → robots.txt shows the version Google has and lets you request a fresh fetch. For the rest of what you can and cannot change on the platform, see what you can and cannot fix on Shopify.

How to undo a mistake

If the file is broken or you are unsure what changed, delete templates/robots.txt.liquid in the code editor. Shopify then serves its default robots.txt again. Deleted templates cannot be restored, which is why you keep a copy of your customised version before editing; paste it into a new template when you are ready.

Common mistakes

• Pasting the live file into the template instead of keeping the loop. The rules stop receiving Shopify's updates.
• Editing the template in an unpublished theme and expecting the live file to change.
• Publishing a new theme without copying robots.txt.liquid across.
• Missing line breaks. Liquid whitespace control ({%- and -%}) can glue two rules onto one line. Check the output.
• Adding Noindex: lines. Google ignores them; see why noindex in robots.txt does not work.
• Blocking products to hide them. They can stay in Google as bare URLs.

Questions people ask

Can you edit robots.txt on Shopify?

Yes, since 2021. Add a robots.txt.liquid template in your theme's code editor: Online Store → Themes → Edit code → Templates → Add a new template → robots.txt. The template prints Shopify's default rules through a Liquid loop, and you add, remove or extend rules around it. You cannot upload a plain robots.txt file.

Why does Shopify block collections in robots.txt?

It blocks only variations of collection pages, not the collections themselves. The default rules stop crawlers from fetching collections sorted in a different order (sort_by) or filtered by several tags at once (+ in the URL), which would otherwise create thousands of near-duplicate pages. The main collection URLs stay open to crawlers.

How do I reset my Shopify robots.txt to the default?

Delete the robots.txt.liquid template from your live theme in the code editor. Shopify then serves its default robots.txt again. A deleted template cannot be recovered, so copy your customised version somewhere safe first if you might want it back.

Should I block AI bots in my Shopify robots.txt?

It depends on your goal. Blocking training crawlers such as GPTBot and CCBot keeps your product pages out of future model training; blocking answer crawlers such as OAI-SearchBot or PerplexityBot keeps you out of AI answers that could link to your store. Add a named group with Disallow: / after the default loop for each crawler you decide to block.