@eldersnake@we.loveprivacy.club ]*}WireGuard (which Tailscale operates and makes management easier) is great!
For an open source alternative to Tailscale, check out Headscale! 👌
@eldersnake@we.loveprivacy.club ]*}WireGuard (which Tailscale operates and makes management easier) is great!
For an open source alternative to Tailscale, check out Headscale! 👌
I must admit Tailscale is really cool and why I haven’t used it before now is beyond me 😅
@bender@twtxt.net Oh gawd 😱
❤️ 🎶: Dropping Rain by Kim Tae Woo
Bit tired myself folks. It’s 00:00 here and I’m going to bed 🛌
On my blog: Free Culture Book Club — Aether Age Codex - Helios, part 2 https://john.colagioia.net/blog/2024/07/27/helios-2.html #freeculture #bookclub
❤️ 🎶: Hold Me Now by Shinae An
@bender@twtxt.net Haha 🤣
@bender@twtxt.net I heard one of the candidates promised to invest 4,000,000 bitcoin 🤣
@bender@twtxt.net it’s very muggy in the Table Tennis hall right now I had to take my jacket off 🤣
Helloo gopherspace. What will you do this weekend?
Olimex Showcases Develoment Board Design Based on ESP32-P4 RISC-V SoC
Olimex Showcases Develoment Board Design Based on ESP32-P4 RISC-V SoC ⌘ Read more
BliKVM Plug-n-Play PCIe with CM4 Now Available at a Reduced Price
BliKVM Plug-n-Play PCIe with CM4 Now Available at a Reduced Price ⌘ Read more
On my blog: Toots 🦣 from 07/22 to 07/26 https://john.colagioia.net/blog/2024/07/26/week.html #linkdump #mastodon #socialmedia #week
Cool! Our park has disappeared again this morning! 😱 Also it was cold outside! 🥶
@lyse@lyse.isobeef.org I’ll fix it tonight Sadly I have to rebuild the index 🤦♂️
@lyse@lyse.isobeef.org This ☝️
@bender@twtxt.net It cost what you mentioned here too, but they had a special discount, and the shop gave 20% discount for all the gear we needed (since I bought the bike + gear for two persons), so all in all I got a really good deal on it. Also interest rate is locked for 3 years (low rate) so that was also a good deal.
I just took the longest time I could on the loan, to have a low price each month, and then i just pay extra whenever I can to get it payed down faster.
Luckily the Marlyn, my son, and my daughter all fit the same gear, so the 3 of them share that, that saved a lot of money too :)
When i met Marlyn (before we had kids) I asked here to go on her first trip with me (I had a bike before I met her back then), and we went on a 2 week ride from Norway to Italy and back (venice) through the alps and all that. Was super fun, and I think it’s pretty cool of her to say yes to a 2 week trip as passenger :)
its not remote… though its on a mountain side where the land grants allowed monopolies to occur. Pretty wild that it happened but only specific vendors have utility right of ways. Its been in litigation with the city for years.
Bought a motorcycle this summer, I did not want two cars, and Marlyn would like to have the car more when I’m at work. So I bought a new cheap motorcycle, KTM Adventure 390.
It’s been 10 years since I last had a motorcycle (back then I had a KTM 990cc).
Here I am with my daughter on the new bike :)
My kids love to go for rides, so does Marlyn as well, so it’s a lot of fun for all of us.
Oh I forgot again 🤦♂️ Last Saturday of the month, so if anyone’s up for a friendly catch up over video tomorrow? Same time, same place 👌
@bender@twtxt.net Weird dunno what to say🤣
@bender@twtxt.net Huh? 🤔
receieveFile()
)? 🤔
Also FWIW this is all my fault for writing shitty vulnerable code 🤣 So blame me! I’m sorry 🙏
receieveFile()
)? 🤔
FWIW I’m still trying to find the the cause of the mult-GB avatars that both @stigatle@yarn.stigatle.no and @abucci@anthony.buc.ci ’s pods were both teying yo download. The flaw has since been fixed in the code but I’m still trying to investigate the source 🤞
@bender@twtxt.net Hehe 🤣
Hmmm something happened last night at ~3am (AEST) that decrased traffic to my pod quite considerably… Hmmm? Anyone have any ideas? 💡
Radxa X4 with Intel Alder Lake-N Processor and 2.5GbE LAN Now Available for Preorder
Radxa X4 with Intel Alder Lake-N Processor and 2.5GbE LAN Now Available for Preorder ⌘ Read more
On my blog: Real Life in Star Trek, Hero Worship https://john.colagioia.net/blog/2024/07/25/hero-worship.html #scifi #startrek #closereading
@bender@twtxt.net haha funny! though i just realized my ISP is the only one with fiber pulled to the property so i would have to get a phone line from them some how. The other ISP in the area is basically a mobile hotspot.
receieveFile()
)? 🤔
We received the abuse report below regarding network abuse from the IP address indicated.
On researching I see that HTTPS (tcp 443) traffic is continuing and originating from you NAT IP address 100.64.x.x
This was further found to be originating from your firewall/router at 192.168.x.x (MAC D8:58:D7:x:x:x).
This abuse is continuing and constitues a violation of
receieveFile()
)? 🤔
@xuu@txt.sour.is wow, not cool.
receieveFile()
)? 🤔
he emailed my ISP about causing logging abuse. This is the only real ISP in my area, its gonna basically send me back to dialup.
receieveFile()
)? 🤔
@xuu@txt.sour.is For what reason?
receieveFile()
)? 🤔
Hey so.. i just got an email from my ISP saying they will terminate my service. Did i break something @abucci@anthony.buc.ci ?
/tmp
is also fine now! Thanks for your help @prologic!
@abucci@anthony.buc.ci No worries! All in the name of better reliability and security 😅
@stigatle@yarn.stigatle.no Thanks! Sooo cold 🥶
receieveFile()
)? 🤔
@stigatle@yarn.stigatle.no no problems 👌 one problem solved at least 🤣
@prologic@twtxt.net sleep well!
receieveFile()
)? 🤔
@stigatle@yarn.stigatle.no @prologic@twtxt.net my /tmp
is also fine now! Thanks for your help @prologic@twtxt.net!
Anyway, I’m gonna have to go to bed… We’ll continue this on the weekend. Still trying to hunt down some kind of suspected mult-GB avatar using @stigatle@yarn.stigatle.no ’s pod’s cache:
$ (echo "URL Bytes"; sort -n -k 2 -r < avatars.txt | head) | column -t
URL Bytes
https://birkbak.neocities.org/avatar.jpg 667640
https://darch.neocities.org/avatar.png 652960
http://darch.dk/avatar.png 603210
https://social.naln1.ca/media/0c4f65a4be32ff3caf54efb60166a8c965cc6ac7c30a0efd1e51c307b087f47b.png 327947
...
But so far nothing much… Still running the search…
receieveFile()
)? 🤔
@prologic@twtxt.net @abucci@anthony.buc.ci my /tmp is fine now, no avatars there. I have to drive my daughter to a birthday party now, but I keep things running and I’ll check when I get back.
receieveFile()
)? 🤔
Out of interest, are you able to block whole ASN(s)? I blocked the entirely of teh AWS and Facebook ASN(s) recently.
receieveFile()
)? 🤔
@abucci@anthony.buc.ci Oh 🤣 Well my IP is a known subnet and static, so if you need to know what it is, Email me 😅
receieveFile()
)? 🤔
@abucci@anthony.buc.ci Seems to be okay now hmmm
@abucci@anthony.buc.ci Hmm I can see your twts on my pod now 🤔
receieveFile()
)? 🤔
@abucci@anthony.buc.ci yeah I can see it :)
@abucci@anthony.buc.ci / @abucci@anthony.buc.ci Any interesting errors pop up in the server logs since the the flaw got fixed (unbounded receieveFile()
)? 🤔
Hmmm 🧐
for url in $(jq -r '.Twters[].avatar' cache.json | sed '/^$/d' | grep -v -E '(twtxt.net|anthony.buc.ci|yarn.stigatle.no|yarn.mills.io)' | sort -u); do echo "$url $(curl -I -s -o /dev/null -w '%header{content-length}' "$url")"; done
...
😅 Let’s see… 🤔
@prologic@twtxt.net will do, thanks for the tip!
@stigatle@yarn.stigatle.no The one you sent is fine. I’m inspecting it now. I’m just saying, do yourself a favor and nuke your pod’s garbage cache 🤣 It’ll rebuild automatically in a much more prestine state.
@prologic@twtxt.net you want a new cache from me - or was the one I sent OK for what you needed?
That was also a source of abuse that also got plugged (being able to fill up the cache with garbage data)
Ooof
$ jq '.Feeds | keys[]' cache.json | wc -l
4402
If you both don’t mind dropping your caches. I would recommend it. Settings -> Poderator Settings -> Refresh cache.
@prologic@twtxt.net No worries, thanks for working on the fix for it so fast :)
@stigatle@yarn.stigatle.no Thank you! 🙏
@prologic@twtxt.net here you go:
https://drive.proton.me/urls/XRKQQ632SG#LXWehEZMNQWF
@stigatle@yarn.stigatle.no Ta. I hope my theory is right 😅
@prologic@twtxt.net thank you. I run it now as you said, I’ll get the files put somewhere shortly.
But just have a look at the yarnd
server logs too. Any new interesting errors? 🤔 No more multi-GB tmp files? 🤔
@stigatle@yarn.stigatle.no You want to run backup_db.sh
and dump_cache.sh
They pipe JSON to stdout and prompt for your admin password. Example:
URL=<your_pod_url> ADMIN=<your_admin_user> ./tools/dump_cache.sh > cache.json
@prologic@twtxt.net so, if I’m correct the dump tool made a pods.txt and a stats.txt file, those are the ones you want? or do you want the output that it spits out in the console window?
Just thinking out loud here… With that PR merged (or if you built off that branch), you might hopefully see new errors popup and we might catch this problematic bad feed in the act? Hmmm 🧐
@slashdot@feeds.twtxt.net I thought Sunday was the hottest day on Earth 🤦♂️ wtf is wrong with Slashdot these days?! 🤣
if we can figure out wtf is going on here and my theory is right, we can blacklist that feed, hell even add it to the codebase as an “asshole”.
@stigatle@yarn.stigatle.no The problem is it’ll only cause the attack to stop and error out. It won’t stop your pod from trying to do this over and over again. That’s why I need some help inspecting both your pods for “bad feeds”.
@prologic@twtxt.net I’m running it now. I’ll keep an eye out for the tmp folder now (I built the branch you have made). I’ll let you know shortly if it helped on my end.
@abucci@anthony.buc.ci / @stigatle@yarn.stigatle.no Please git pull
, rebuild and redeploy.
There is also a shell script in ./tools
called dump_cache.sh
. Please run this, dump your cache and share it with me. 🙏
I’m going to merge this…
@abucci@anthony.buc.ci Yeah I’ve had to block entire ASN(s) recently myself from bad actors, mostly bad AI bots actually from Facebook and Caude AI
Or if y’all trust my monkey-ass coding skillz I’ll just merge and you can do a git pull
and rebuild 😅
@stigatle@yarn.stigatle.no / @abucci@anthony.buc.ci My current working theory is that there is an asshole out there that has a feed that both your pods are fetching with a multi-GB avatar URL advertised in their feed’s preamble (metadata). I’d love for you both to review this PR, and once merged, re-roll your pods and dump your respective caches and share with me using https://gist.mills.io/
@prologic@twtxt.net yeah I still do have that issue, I compiled latest main, did not apply any patches or anything like that.
@stigatle@yarn.stigatle.no I’m wondering whether you’re having the same issue as @abucci@anthony.buc.ci still? mulit-GB yarnd-avatar-*1
files piling up in /tmp/
? 🤔
@prologic@twtxt.net yeah, I ran out of space again. also have the activitypub stuff turned off (just so you know).
watch -n 60 rm -rf /tmp/yarn-avatar-*
in a tmux
because all of a sudden, without warning, yarnd
started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554
into /tmp
, which filled up the entire disk and started crashing other services.
@abucci@anthony.buc.ci So… The only way I see this happening at all is if your pod is fetching feeds which have multi-GB sized avatar(s) in their feed metadata. So the PR I linked earlier will plug that flaw. But now I want to confirm that theory. Can I get you to dump your cache to JSON for me and share it with me?
@abucci@anthony.buc.ci Yeah that should be okay, you get so much crap on the web 🤦♂️
watch -n 60 rm -rf /tmp/yarn-avatar-*
in a tmux
because all of a sudden, without warning, yarnd
started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554
into /tmp
, which filled up the entire disk and started crashing other services.
@abucci@anthony.buc.ci sift
is a tool I use for grep/find, etc.
What would you like to know about the files?
Roughly what their contents are. I’ve been reviewing the code paths responsible and have found a flaw that needs to be fixed ASAP.
Here’s the PR: https://git.mills.io/yarnsocial/yarn/pulls/1169
Monday Was Hottest Recorded Day on Earth: ‘Uncharted Territory’
World temperature reached the hottest levels ever measured on Monday, beating the record that was set just one day before, data suggests. From a report: Provisional data published on Wednesday by the Copernicus Climate Change Service, which holds data that stretches back to 1940, shows that the global surface air temperature reached 62.87F (17.15C), co … ⌘ Read more
@abucci@anthony.buc.ci I believe you are correct.
@abucci@anthony.buc.ci That’s fucking insane 😱 I know what code-paths is triggering this, but need to confirm a few other things… Some correlation with logs would also help…
watch -n 60 rm -rf /tmp/yarn-avatar-*
in a tmux
because all of a sudden, without warning, yarnd
started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554
into /tmp
, which filled up the entire disk and started crashing other services.
Do you happen to have the activitypub
feature turned on btw? In fact could you just list out what features you have enabled please? 🙏
watch -n 60 rm -rf /tmp/yarn-avatar-*
in a tmux
because all of a sudden, without warning, yarnd
started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554
into /tmp
, which filled up the entire disk and started crashing other services.
These should be getting cleaned up, but I’m very concerned about the sizes of these 🤔
watch -n 60 rm -rf /tmp/yarn-avatar-*
in a tmux
because all of a sudden, without warning, yarnd
started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554
into /tmp
, which filled up the entire disk and started crashing other services.
Hah 😈
prologic@JamessMacStudio
Fri Jul 26 00:22:44
~/Projects/yarnsocial/yarn
(main) 0
$ sift 'yarnd-avatar-*'
internal/utils.go:666: tf, err := receiveFile(res.Body, "yarnd-avatar-*")
@abucci@anthony.buc.ci Don’t suppose you can inspect one of those files could you? Kinda wondering if there’s some other abuse going on here that I need to plug? 🔌
watch -n 60 rm -rf /tmp/yarn-avatar-*
in a tmux
because all of a sudden, without warning, yarnd
started throwing hundreds of gigabytes of files with names like yarn-avatar-62582554
into /tmp
, which filled up the entire disk and started crashing other services.
@abucci@anthony.buc.ci Hmm that’s a bit weird then. Lemme have a poke.
Hmm remove the cpu limits on this pod, not even sure why I had ‘em set tbh, we decided at my day job that setting cpu limits on containers is a bit of a silly idea too. Anyway, pod should be much snappier now 😅