StoneParty&Corp SEO·AEO·GEO Guide

Q19( Answer Engine Optimization )

nosnippet: Block Snippets and AI Answers

Question: Can I stop my content from being used in search snippets and AI answers?

Quick answer

You can control much of how your content is used in search snippets and AI answers, but each method covers a different scope. Snippets in Google Search and AI Overviews and AI Mode are controlled with nosnippet, data-nosnippet, max-snippet, and noindex, and collection for AI model training is controlled with per-bot robots.txt rules for bots such as GPTBot and ClaudeBot. However, following robots.txt is voluntary for crawlers, and blocking can also reduce your visibility.

Key takeaways

  • nosnippet blocks all snippets for the page.
  • data-nosnippet excludes only the specified text from snippets.
  • Google-Extended does not affect inclusion or ranking in Google Search.
  • GPTBot is the training bot, and OAI-SearchBot is the ChatGPT search bot.
  • Following robots.txt rules is voluntary for crawlers.

How do you block Google Search snippets?

Google Search snippets are controlled with three settings: nosnippet, data-nosnippet, and max-snippet. nosnippet blocks all snippets for the page, data-nosnippet excludes only the text you specify, and max-snippet limits the length of the snippet.

To keep only certain paragraphs, not the whole page, from being excerpted, the right method is to add the data-nosnippet attribute to those parts. If you want to reduce only featured snippets, the guidance is to set max-snippet to a short length.

To remove the page itself from search results, use noindex. Unlike snippet controls, noindex removes search visibility altogether, so use each according to its purpose.

Scope of each control method (as of writing in 2026, may change)
MethodApplies toEffect
nosnippetGoogle Search (including AI features)Blocks all snippets for the page
data-nosnippetGoogle Search (including AI features)Excludes only the specified text from snippets
max-snippetGoogle Search (including AI features)Limits snippet length; a short value reduces featured snippets
noindexGoogle Search (including AI features)Removes the page from search results
Blocking Google-ExtendedGemini training and groundingNo effect on inclusion or ranking in Google Search
Blocking GPTBotCollection for OpenAI model trainingSignals that the content should not be used for training
Blocking OAI-SearchBotChatGPT searchNot shown in search answers (may still appear as a navigational link)
Blocking ClaudeBotCollection for Anthropic model trainingSignals exclusion from future training data

Can I block use only in Google AI Overviews and AI Mode?

AI features in Google Search are controlled with the same methods as regular search. Google says control over AI Overviews and AI Mode is done through robots.txt rules for Googlebot and through nosnippet, data-nosnippet, max-snippet, and noindex.

Google-Extended is often confused with these. Google-Extended is a control token for Gemini model training and grounding, and it does not affect inclusion or ranking in Google Search. It has no separate HTTP user agent and is used only as a robots.txt token.

In other words, blocking Google-Extended is not a way to control AI Overviews in Google Search. Also keep in mind that if you use snippet rules to reduce use in AI features, snippets in regular search results may be affected too.

How does this apply to ChatGPT and Claude?

Set separate robots.txt rules for each bot name the companies have published. Training bots and search bots are separate, so you can set things up so your site appears in search but is not used for training.

For OpenAI, GPTBot is the bot that collects content that may be used for model training, and OAI-SearchBot is the bot for showing sites in ChatGPT search results. The two settings are independent, so you can put 'User-agent: GPTBot' and 'Disallow: /' in robots.txt and still allow OAI-SearchBot.

Anthropic distinguishes ClaudeBot for training, Claude-SearchBot for search, and Claude-User for user requests, and says its bots follow robots.txt. Bot names and behavior are as of writing (2026) and may change, so check each company's official documentation.

Does blocking with robots.txt block everything?

No, it cannot be considered a complete block. Following robots.txt rules is voluntary for crawlers, and the rules may not apply to agents that fetch pages at a user's request.

OpenAI explains that robots.txt rules may not apply to ChatGPT-User because its actions are initiated by users, and Perplexity says Perplexity-User generally ignores robots.txt rules.

Google also explains that robots.txt is not a way to hide a page from Google. If the content must not be public, the right approach is to use noindex or protect it with a password.

What should you weigh before blocking?

Blocking is a decision that trades away visibility, so first identify whether what you want to stop is snippet excerpts, citation in AI answers, or model training.

Answer engine optimization (AEO) and generative engine optimization (GEO) aim to get content used in answers, which is the opposite direction from blocking. Visits earned through search engine optimization (SEO) may also be affected, so the common approach is to apply restrictions only to paid content or certain sensitive areas and leave the rest open.

Frequently asked questions

What is the difference between noindex and blocking with robots.txt?

robots.txt is a file that tells crawlers which URLs they can access, while noindex is a rule that removes a page from search results. Google explains that robots.txt is mainly for avoiding overloading a site with requests, so to keep a page out of search, use noindex or password protection.

When do changes to these settings take effect?

It differs by service. Google says changes can take from a few hours to several months, and the rules take effect after the page is crawled again. For AI services, it is best to check each company's official documentation for how long changes take.

Can content that has already been used for training be taken back?

Blocking with robots.txt is a statement of intent about future collection. Anthropic describes blocking ClaudeBot as a signal to exclude content from future training data. How already collected content is handled may differ by company, so check each company's policy.

Key entities in this article

Google ( Organization )
The company behind Google Search, AI Overviews and AI Mode, which publishes official SEO guidance through Search Central. Official source: Google
AI Overviews ( Concept or service )
The Google Search feature in which AI summarizes multiple sources and shows them with links. Official source: AI Overviews
AI Mode ( Concept or service )
The conversational AI search experience in Google Search for follow-up questions and complex comparisons. Official source: AI Mode
robots.txt ( Concept or service )
The file that tells crawlers which URLs on a site they may access, standardized as RFC 9309. Official source: robots.txt
Gemini ( Concept or service )
Google's generative AI model family and conversational AI service. Official source: Gemini
ChatGPT ( Concept or service )
OpenAI's conversational AI service, which includes a search feature that answers with cited web results. Official source: ChatGPT
Claude ( Concept or service )
The conversational AI service made by Anthropic. Official source: Claude
OpenAI ( Organization )
The US AI company behind ChatGPT, which operates the GPTBot, OAI-SearchBot and ChatGPT-User crawlers. Official source: OpenAI
Anthropic ( Organization )
The US AI company behind Claude, which operates the ClaudeBot, Claude-User and Claude-SearchBot crawlers. Official source: Anthropic
Perplexity ( Organization )
The company behind an AI search service that shows its sources; it operates PerplexityBot and Perplexity-User. Official source: Perplexity
Answer engine optimization (AEO) ( Concept or service )
Structuring content and information so search engines and AI can present it as the single answer to a question.
Generative engine optimization (GEO) ( Concept or service )
The work of making content and brands more visible and more often cited in answers that generative AI builds from multiple sources. Official source: Generative engine optimization (GEO)

Sources

  1. Google Search Central: Robots meta tag
  2. Google Search Central: AI features and your website
  3. OpenAI: Overview of OpenAI crawlers
  4. Anthropic: Web crawler information

All questions in Answer Engine Optimization

  1. What is a featured snippet?
  2. How do you write content that AI can easily use as an answer?
  3. Does FAQ structured data still work?
  4. How do you find the questions customers actually ask?
  5. Can I stop my content from being used in search snippets and AI answers?
  6. Do site visits fall as zero-click searches increase?
View Answer Engine Optimization