There is a bug in yarnd that’s been around for awhile and is still present in the current version I’m running that lets a person hit a constructed URL like

YOUR_POD/external?nick=lovetocode999&uri=https://socialmphl.com/story19510368/doujin

and see a legitimate-looking page on YOUR_POD, with an HTTP code 200 (success). From that fake page you can even follow an external feed. Try it yourself, replacing “YOUR_POD” with the URL of any yarnd pod you know. Try following the feed.

I think URLs like this should return errors. They should not render HTML, nor produce legitimate-looking pages. This mechanism is ripe for DDoS attacks. My pod gets roughly 70,000 hits per day to URLs like this. Many are porn or other types of content I do not want. At this point, if it’s not fixed soon I am going to have to shut down my pod. @prologic@twtxt.net please have a look.

@abucci@anthony.buc.ci Hmmm I’ll have a look at this today, hopefully. 🤞 Thanks for bringing this up! 🙇‍♂️

@abucci@anthony.buc.ci I’ve fixed this and pushed a commit to main. Will test it on my pod and see how it goes. Basically reporting an error if the feed isn’t in the cache and you’re unauthenticated (anonymous).

Now responds with an error page; but still a 200 OK which I’m not entirely sure I agree with or am happy with? Hmm

I’ve pushed a slightly improved version of this that will return a HTTP 404 Not Found if the UserAgent is determined to NOT be a Browser.

@prologic@twtxt.net hmm… I think we should do better than this. It is an improvement though. Ideally we check that URI to make sure it is a twtxt.txt (contents, or otherwise). If it isn’t, don’t link, don’t follow, don’t “create” an empty page. If it is not twtxt, it shouldn’t be on twtxt. :-)

@bender@twtxt.net The problem with this is we just don’t know until we try. That’s why if the external feed you’re looking for isn’t found in the cache, it’ll try to fetch it in the background. It’s a bit of a sucky UX really, but its better than the experience of “waiting, waiting waiting and then timeout”.

I’m happy with the current implementation though, because the only reason you should be hitting the external profile endpoint at all is a) you’re logged in and happen to click on someone’s profile that is external to the pod or b) you’re anonymous and just clicking through the frontpage (see a)

@prologic@twtxt.net sounds fair. Let’s see how it works for @abucci@anthony.buc.ci. Speedy fix, that’s awesome! :-)

@quark@ferengi.one Thanks! 😅

@abucci@anthony.buc.ci It’s good enough IMO 🤞

@abucci@anthony.buc.ci This is already in place. It will error, return 404 Feed Not Found for non-browsers and external feeds are never fetched (unless you are an authenticated/valid user of the pod) – I patched that hole a while ago, because I already picked up it was being abused by bots 🤖

@abucci@anthony.buc.ci Blah my cache was poisoned 🤦‍♂️ it’s fine now! And this is no monger possible to do now.

@abucci@anthony.buc.ci Bo worries! If you curl it too it’ll return a proper 494 👌 Should make bots go away 🤞

It appears to be working to 👌

silly bots 🙄

Hmmm

@abucci@anthony.buc.ci You don’t actually appear to be running that sha hmmm? 🤔

@abucci@anthony.buc.ci Hmmm weird 🤔

How did you nuke your cache?

@abucci@anthony.buc.ci nuke the cache file before starring

But this is super weird, should behave the same as my pod 🤦‍♂️

A stopgap setting that would let me stop all calls to /external matching a particular pattern (like this damn lovetocode999 nick) would do the job. Given the potential for abuse of that endpoint, having more moderation control over what it can do is probably a good idea.