# Allow all crawlers User-agent: * Allow: / # Disallow admin and API routes Disallow: /admin Disallow: /api/ Disallow: /horizon # Disallow pages containing personal consumer data Disallow: /auto/mf/ Disallow: /home/mf/ Disallow: /auto/quotes/ Disallow: /home/quotes/