↳
In-reply-to
»
I just banned 41 bad user agents from accessing any of my services. 😱
⤋ Read More
@doesnm@doesnm.p.psf.lt No. I generally don’t put up any robots.txt files at all really, because they mostly get ignored. I don’t generally mind if “normal” web crawlers crawl things. But LLM(s) can go fuck themselves 🤣