↳
In-reply-to
»
It would appear that Google's web crawlers are ignoring the
⤋ Read More
robots.txt that I have on https://git.mills.io/robots.txt with content:
@prologic@twtxt.net Have you tried Google’s robots.txt report? https://support.google.com/webmasters/answer/6062598?hl=en . I would expect Google to be pretty good about this sort of thing. If you have the energy to dig into it and, for example, post on support.google.com, I’d be curious to hear what you find out.