@bmallred@staystrong.run Surprisingly, my

User-agent: *
Disallow: /

seems to work. Or maybe those bastards change their user agent and claim to be someone nice. In any case, I just added a bunch of

location = /robots.txt {
    add_header Content-Type text/plain;
    return 200 "User-agent: *\nDisallow: /\n";
}

in my nginx config. No need for any bot to visit, crawl and index most of my sites.

⤋ Read More