# ============================================================ # Artizon — robots.txt # Objectif : diriger le crawl vers les pages utiles (blog, # villes, métiers) et couper l'exploration inutile # (espaces privés, URLs paramétrées, bots agressifs). # ============================================================ # ---------- Règles communes (tous les robots) ---------- User-agent: * Allow: / # Blog : contenus utiles explicitement autorisés Allow: /blog Allow: /blog/ Allow: /blog/categorie/ Allow: /blog/tag/ # Blog : variantes inutiles (pagination JS, tri, recherche, impression) Disallow: /blog/*? Disallow: /blog/*& Disallow: /blog/*/amp Disallow: /blog/preview Disallow: /blog/draft Disallow: /blog/brouillon # URLs paramétrées (duplication + gaspillage de budget de crawl) Disallow: /*?utm_ Disallow: /*?gclid= Disallow: /*?fbclid= Disallow: /*?ref= Disallow: /*?src= Disallow: /*?token= Disallow: /*?t= Disallow: /*&utm_ Disallow: /*?q= Disallow: /*?search= # Espaces privés / transactionnels Disallow: /cgu-pro Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /disavow-artizon.txt Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /embed-chat Disallow: /integrer-chat Disallow: /installer-app Disallow: /reset-password Disallow: /auth/ Disallow: /avis/ Disallow: /debug/ Disallow: /tech/ Disallow: /mon-espace Disallow: /notifications Disallow: /relance/ Disallow: /retrouver-chat Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /embed-form Disallow: /embed/ # ---------- Googlebot (mêmes restrictions, sinon le groupe * est ignoré) ---------- User-agent: Googlebot Allow: / Allow: /blog Allow: /blog/ Allow: /blog/categorie/ Allow: /blog/tag/ Disallow: /blog/*? Disallow: /blog/*& Disallow: /blog/preview Disallow: /blog/draft Disallow: /blog/brouillon Disallow: /*?utm_ Disallow: /*?gclid= Disallow: /*?fbclid= Disallow: /*?ref= Disallow: /*?src= Disallow: /*?token= Disallow: /*?q= Disallow: /*?search= Disallow: /cgu-pro Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /embed-chat Disallow: /integrer-chat Disallow: /installer-app Disallow: /reset-password Disallow: /auth/ Disallow: /avis/ Disallow: /debug/ Disallow: /tech/ Disallow: /mon-espace Disallow: /notifications Disallow: /relance/ Disallow: /retrouver-chat Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /embed-form Disallow: /embed/ # Googlebot-Image : visuels du blog et des métiers uniquement User-agent: Googlebot-Image Allow: / Disallow: /admin/ Disallow: /espace-technicien/ Disallow: /embed/ # ---------- Bingbot ---------- User-agent: Bingbot Allow: / Allow: /blog/categorie/ Allow: /blog/tag/ Disallow: /blog/*? Disallow: /*?utm_ Disallow: /*?gclid= Disallow: /*?fbclid= Disallow: /*?ref= Disallow: /*?q= Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /auth/ Disallow: /avis/ Disallow: /tech/ Disallow: /relance/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe # ---------- Réseaux sociaux (aperçus de partage) ---------- User-agent: Twitterbot Allow: / User-agent: facebookexternalhit Allow: / User-agent: LinkedInBot Allow: / # ---------- LLM / AI crawlers ---------- # Les groupes ci-dessous sont autonomes : un robot nommé ici ignore # totalement le groupe « User-agent: * ». Les exclusions privées y sont # donc recopiées à l'identique. robots.txt n'est PAS un contrôle d'accès : # la protection des données privées repose sur l'authentification et les # règles d'accès en base, pas sur ce fichier. # Le consentement d'entraînement (GPTBot, CCBot, Google-Extended, # Applebot-Extended) reste inchangé : seules les zones privées sont exclues. User-agent: GPTBot Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: OAI-SearchBot Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: ChatGPT-User Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: PerplexityBot Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: Perplexity-User Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: ClaudeBot Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: Claude-Web Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: Google-Extended Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: Applebot-Extended Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: Applebot Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= User-agent: CCBot Allow: / Disallow: /admin Disallow: /admin/ Disallow: /pro/ Disallow: /espace-technicien Disallow: /espace-technicien/ Disallow: /mes-conversations Disallow: /mes-demandes Disallow: /mon-espace Disallow: /notifications Disallow: /confirmer/ Disallow: /signaler-probleme/ Disallow: /relance/ Disallow: /retrouver-chat Disallow: /avis/ Disallow: /auth/ Disallow: /reset-password Disallow: /tech/ Disallow: /debug/ Disallow: /embed/ Disallow: /embed-chat Disallow: /embed-form Disallow: /integrer-chat Disallow: /installer-app Disallow: /paiement-succes Disallow: /paiement-annule Disallow: /statut-paiement Disallow: /stripe-return Disallow: /unsubscribe Disallow: /cgu-pro Disallow: /*?token= # ---------- Crawlers SEO/scraping gourmands : bridés ---------- User-agent: AhrefsBot Crawl-delay: 10 Disallow: /blog/tag/ User-agent: SemrushBot Crawl-delay: 10 Disallow: /blog/tag/ User-agent: MJ12bot Disallow: / User-agent: DotBot Disallow: / User-agent: Bytespider Disallow: / User-agent: PetalBot Crawl-delay: 10 # ---------- Sitemaps & flux ---------- Sitemap: https://artizon.fr/sitemap.xml Sitemap: https://artizon.fr/sitemap-blog.xml Sitemap: https://artizon.fr/sitemap-blog-categories.xml Sitemap: https://artizon.fr/sitemap-blog-tags.xml Sitemap: https://artizon.fr/rss.xml # ---------- Corpus pour moteurs de reponse (GEO) ---------- # https://artizon.fr/llms.txt : index lisible par les LLM # https://artizon.fr/llms-full.txt : corpus complet (faits + Q/R sourcees)