↳
In-reply-to
»
🤣 Indeed. It's like using disallow is only pointing them to look for more stuff to index. Those few kB's for a twtxt file, meh.. shouldnt even be a discussion.
⤋ Read More
@Codebuzz@www.codebuzz.nl That’s what I figure as well, thus my robot.txt looks like that:
User-agent: *
Disallow: /
Allow: /$