@bmallred@staystrong.run Surprisingly, my
User-agent: *
Disallow: /
seems to work. Or maybe those bastards change their user agent and claim to be someone nice. In any case, I just added a bunch of
location = /robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *\nDisallow: /\n";
}
in my nginx config. No need for any bot to visit, crawl and index most of my sites.