# robots.txt # All public pages, images, CSS and JavaScript are open to search engines and to AI search, # retrieval and model-training crawlers. Only form endpoints and admin paths are excluded. # robots.txt is a request to crawlers, not access control. # AI search, retrieval and training crawlers (explicitly allowed) # OpenAI User-agent: GPTBot User-agent: OAI-SearchBot User-agent: ChatGPT-User # Anthropic User-agent: ClaudeBot User-agent: Claude-SearchBot User-agent: Claude-User # Google (Google-Extended is a control token, not a crawler) User-agent: Googlebot User-agent: Google-Extended # Meta User-agent: meta-externalagent User-agent: meta-webindexer User-agent: meta-externalfetcher # Apple (Applebot-Extended is a control token, not a crawler) User-agent: Applebot User-agent: Applebot-Extended # Common Crawl User-agent: CCBot # Perplexity User-agent: PerplexityBot User-agent: Perplexity-User # Amazon User-agent: Amazonbot User-agent: Amzn-SearchBot User-agent: Amzn-User # ByteDance User-agent: Bytespider # Mistral AI User-agent: MistralAI-Training User-agent: MistralAI-Index User-agent: MistralAI-User Allow: / Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Disallow: /*?submitted= # Every other crawler User-agent: * Allow: / Disallow: /wp-admin/ Allow: /wp-admin/admin-ajax.php Disallow: /*?submitted= Sitemap: https://www.egaragedoors.com/sitemap.xml