Browse Source

robots.txt: opt out of search engines for now

remotes/blesmrt/master
Mikaela Suomalainen 5 months ago
parent
commit
c90a96bd8a
Signed by: Mikaela Suomalainen <mikaela+git@mikaela.info> GPG Key ID: 0C207F07B2F32B67
1 changed files with 2 additions and 23 deletions
  1. +2
    -23
      robots.txt

+ 2
- 23
robots.txt View File

@@ -1,26 +1,5 @@
User-agent: *
Allow: /
Disallow: /404
Noindex: /404
Disallow: /bot
Noindex: /bot
Disallow: /channel
Noindex: /channel
Disallow: /links2
Noindex: /links2
Disallow: /irc/bot
Noindex: /irc/bot
Disallow: /irc/channel
Noindex: /irc/channel
Disallow: /irc/znc
Noindex: /irc/znc
Disallow: /r/
Noindex: /r/
Disallow: /or/
Noindex: /or/
Disallow: /ir/
Noindex: /ir/
Disallow: /reflinks
Noindex: /reflinks
Disallow: /
Noindex: /
Sitemap: https://mikaela.info/sitemap.xml
Sitemap: https://mikaela.info/sitemaps.xml

Loading…
Cancel
Save