User-agent: * Allow: / # Blog search result pages. Every one of them canonicalises to /blog (or to the # tag archive it filters), so they are already folded rather than indexed. This # stops them being fetched at all, which canonicalisation cannot do. # # robots.txt rather than a `noindex` meta on those pages, on purpose: combining # `noindex` with a canonical pointing elsewhere risks the directive being applied # to the canonical target, and losing /blog itself would be far worse than a # few crawled search URLs. # # `*` and `$` in a path are extensions to the original standard, but every major # crawler supports them. Disallow: /*?q= # Locale addresses of English-only pages. They serve the English document in # translated chrome and canonicalise to the unprefixed URL, so they exist for # readers and not for crawlers. Generated from src/lib/locale.ts. Disallow: /fr/blog$ Disallow: /fr/blog/ Disallow: /fr/blog? Disallow: /fr/help$ Disallow: /fr/help/ Disallow: /fr/help? Disallow: /fr/legal-notices$ Disallow: /fr/legal-notices/ Disallow: /fr/legal-notices? Disallow: /fr/privacy$ Disallow: /fr/privacy/ Disallow: /fr/privacy? Disallow: /fr/security$ Disallow: /fr/security/ Disallow: /fr/security? Disallow: /fr/status$ Disallow: /fr/status/ Disallow: /fr/status? Disallow: /fr/terms$ Disallow: /fr/terms/ Disallow: /fr/terms? Disallow: /fr/vs$ Disallow: /fr/vs/ Disallow: /fr/vs? Disallow: /fr/domain-expiry-checker$ Disallow: /fr/domain-expiry-checker/ Disallow: /fr/domain-expiry-checker? Disallow: /de/blog$ Disallow: /de/blog/ Disallow: /de/blog? Disallow: /de/help$ Disallow: /de/help/ Disallow: /de/help? Disallow: /de/legal-notices$ Disallow: /de/legal-notices/ Disallow: /de/legal-notices? Disallow: /de/privacy$ Disallow: /de/privacy/ Disallow: /de/privacy? Disallow: /de/security$ Disallow: /de/security/ Disallow: /de/security? Disallow: /de/status$ Disallow: /de/status/ Disallow: /de/status? Disallow: /de/terms$ Disallow: /de/terms/ Disallow: /de/terms? Disallow: /de/vs$ Disallow: /de/vs/ Disallow: /de/vs? Disallow: /de/domain-expiry-checker$ Disallow: /de/domain-expiry-checker/ Disallow: /de/domain-expiry-checker? Disallow: /nl/blog$ Disallow: /nl/blog/ Disallow: /nl/blog? Disallow: /nl/help$ Disallow: /nl/help/ Disallow: /nl/help? Disallow: /nl/legal-notices$ Disallow: /nl/legal-notices/ Disallow: /nl/legal-notices? Disallow: /nl/privacy$ Disallow: /nl/privacy/ Disallow: /nl/privacy? Disallow: /nl/security$ Disallow: /nl/security/ Disallow: /nl/security? Disallow: /nl/status$ Disallow: /nl/status/ Disallow: /nl/status? Disallow: /nl/terms$ Disallow: /nl/terms/ Disallow: /nl/terms? Disallow: /nl/vs$ Disallow: /nl/vs/ Disallow: /nl/vs? Disallow: /nl/domain-expiry-checker$ Disallow: /nl/domain-expiry-checker/ Disallow: /nl/domain-expiry-checker? Disallow: /it/blog$ Disallow: /it/blog/ Disallow: /it/blog? Disallow: /it/help$ Disallow: /it/help/ Disallow: /it/help? Disallow: /it/legal-notices$ Disallow: /it/legal-notices/ Disallow: /it/legal-notices? Disallow: /it/privacy$ Disallow: /it/privacy/ Disallow: /it/privacy? Disallow: /it/security$ Disallow: /it/security/ Disallow: /it/security? Disallow: /it/status$ Disallow: /it/status/ Disallow: /it/status? Disallow: /it/terms$ Disallow: /it/terms/ Disallow: /it/terms? Disallow: /it/vs$ Disallow: /it/vs/ Disallow: /it/vs? Disallow: /it/domain-expiry-checker$ Disallow: /it/domain-expiry-checker/ Disallow: /it/domain-expiry-checker? Disallow: /es/blog$ Disallow: /es/blog/ Disallow: /es/blog? Disallow: /es/help$ Disallow: /es/help/ Disallow: /es/help? Disallow: /es/legal-notices$ Disallow: /es/legal-notices/ Disallow: /es/legal-notices? Disallow: /es/privacy$ Disallow: /es/privacy/ Disallow: /es/privacy? Disallow: /es/security$ Disallow: /es/security/ Disallow: /es/security? Disallow: /es/status$ Disallow: /es/status/ Disallow: /es/status? Disallow: /es/terms$ Disallow: /es/terms/ Disallow: /es/terms? Disallow: /es/vs$ Disallow: /es/vs/ Disallow: /es/vs? Disallow: /es/domain-expiry-checker$ Disallow: /es/domain-expiry-checker/ Disallow: /es/domain-expiry-checker? Disallow: /pt/blog$ Disallow: /pt/blog/ Disallow: /pt/blog? Disallow: /pt/help$ Disallow: /pt/help/ Disallow: /pt/help? Disallow: /pt/legal-notices$ Disallow: /pt/legal-notices/ Disallow: /pt/legal-notices? Disallow: /pt/privacy$ Disallow: /pt/privacy/ Disallow: /pt/privacy? Disallow: /pt/security$ Disallow: /pt/security/ Disallow: /pt/security? Disallow: /pt/status$ Disallow: /pt/status/ Disallow: /pt/status? Disallow: /pt/terms$ Disallow: /pt/terms/ Disallow: /pt/terms? Disallow: /pt/vs$ Disallow: /pt/vs/ Disallow: /pt/vs? Disallow: /pt/domain-expiry-checker$ Disallow: /pt/domain-expiry-checker/ Disallow: /pt/domain-expiry-checker? Sitemap: https://okokumo.com/sitemap.xml