GEO / AI Search
GPTBot
Definition
What is GPTBot?
OpenAI's web crawler used to populate ChatGPT's training data and live citations. Identified by the User-Agent string GPTBot/1.0. Sites that block it via robots.txt are excluded from ChatGPT-generated answers.
Related terms
Cross-linked concepts
- ClaudeBotAnthropic's web crawler for Claude. Identified by User-Agent ClaudeBot/1.0 and anthropic-ai. Allowing it in robots.txt is required to be cited in Claude-generated answers.
- PerplexityBotPerplexity AI's web crawler. Two variants: PerplexityBot (training) and Perplexity-User (live citation lookup). Citation rates correlate strongly with named-author bylines, visible publish dates, and high domain authority.
- robots.txtA plain-text file at the domain root that tells crawlers which paths they can access. Can target specific User-Agent strings (Googlebot, GPTBot, ClaudeBot). Blocking AI crawlers here removes the site from AI-generated answers.
Same category
More GEO / AI Search terms
Want to put GPTBot to work?
We run GEO / AI Search-led engagements every week. A 20-minute call gets you a read on whether gptbot is the right next move for your stack.