It would appear that Google’s web crawlers are ignoring the robots.txt that I have on https://git.mills.io/robots.txt with content:

User-agent: *
Disallow: /

Evidence attached (see screenshots): – I think its the the Small Web community band together and file a class action suit(s) against Microsoft.com Google.com and any other assholes out there (OpenAI?) that violate our rights and ignore requests to be “polite” on the web. Thoughts? 💭

@prologic@twtxt.net Have you tried Google’s robots.txt report? https://support.google.com/webmasters/answer/6062598?hl=en . I would expect Google to be pretty good about this sort of thing. If you have the energy to dig into it and, for example, post on support.google.com, I’d be curious to hear what you find out.

@falsifian@www.falsifian.org In the process of 🤞