# robots.txt — indexing guidance for COMPLIANT crawlers (Google, Bing, social, etc.). # NOTE: this does NOT block scrapers or CARFAX — robots.txt is advisory only and # adversarial bots ignore it. Real bot-blocking is enforced at Cloudflare. User-agent: * Allow: / # API and shared report links stay uncrawled. The private app pages (dashboard, history, admin, # success) are NOT disallowed on purpose: they send noindex (meta tag + X-Robots-Tag), and a crawler # can only obey noindex on a page it is allowed to fetch. Blocking them here got the bare URLs indexed. Disallow: /api/ Disallow: /view/ Sitemap: https://cheapestcarfax.com/sitemap.xml