# AI and training crawlers: do not crawl search-result pages. User-agent: GPTBot User-agent: ChatGPT-User User-agent: OAI-SearchBot User-agent: ClaudeBot User-agent: Claude-Web User-agent: anthropic-ai User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Google-Extended User-agent: Applebot-Extended User-agent: CCBot User-agent: Bytespider User-agent: Amazonbot User-agent: Meta-ExternalAgent User-agent: meta-externalfetcher Disallow: /search Disallow: /api/ Disallow: /admin/ Disallow: /reports/ # Allow normal search engines to index the site. User-agent: * Allow: / # Disallow admin and API routes Disallow: /admin/ Disallow: /api/ Disallow: /reports/ # Sitemap location Sitemap: https://goonchan.org/sitemap.xml