prologic

twtxt.net

Problems are Solved by Method\" 🇦🇺👨‍💻👨‍🦯🏹♔ 🏓⚯ 👨‍👩‍👧‍👧🛥 -- James Mills (operator of twtxt.net / creator of Yarn.social 🧶)

How in da fuq do you actually make these fucking useless AI bots go way?

proxy-1:~# jq '. | select(.request.remote_ip=="4.227.36.76")' /var/log/caddy/access/mills.io.log | jq -s '. | last' | caddy-log-formatter -
4.227.36.76 - [2025-01-05 04:05:43.971 +0000] "GET /external?aff-QNAXWV=&f=mediaonly&f=noreplies&nick=g1n&uri=https%3A%2F%2Fmy-hero-ultra-impact-codes.linegames.org HTTP/2.0" 0 0
proxy-1:~# date
Sun Jan  5 04:05:49 UTC 2025

😱

⤋ Read More
In-reply-to » @andros What do you mean by API? yarnd (which powers Yarn.social pods like twtxt.net) does have an API, however that API is designed for clients to interact with the pod and the user's account and feed. e.g: there is a command-line client called yarnc and I used to maintain a mobile native app (using Flutter).

@doesnm@doesnm.p.psf.lt It is the same API that yarnc the command-line client uses.

⤋ Read More
In-reply-to » @prologic Is it possible to interact with twtxt.net from outside? For example, an search API

@andros@twtxt.andros.dev What do you mean by API? yarnd (which powers Yarn.social pods like twtxt.net) does have an API, however that API is designed for clients to interact with the pod and the user’s account and feed. e.g: there is a command-line client called yarnc and I used to maintain a mobile native app (using Flutter).

What use-case did you have in mind?

⤋ Read More

I am now proud to say, that as of this moment, I am off of Clownflare 🤣 Still using Cloudflare for DNS, but no longer proxying through their services or terminating TLS at their edge. Instead, all my sites and services now terminate TLS on my own edge proxy running Caddy+Wireguard (so all ingress is actually egress 🤣) 🥳 #Clownflare #Cloudflare

⤋ Read More
In-reply-to » @prologic It's hosted at home on an computer I didn’t use anymore. It worked well for a few months, and since maybe the beginning of December, it begun to be very slow. But like I said, I have no time for that now, but if I have questions when I’ll look, I’ll think of you 😅 (but I was thinking about installing a new OS before these problems, I may just do that).

@emmanuel@wald.ovh Btw I already figured out why accessing your web server is slow:

$ host wald.ovh
wald.ovh has address 86.243.228.45
wald.ovh has address 90.19.202.229

wald.ovh has 2 IPv4 addresses, one of which is dead and doesn’t respond.. That’s why accessing your website is so slow as depending on client and browser behaviors one of two things may happen 1) a random IP is chosen and ½ the time the wrong one is picked or 2) both are tried in some random order and ½ the time its slow because the broken one is picked.

If you don’t know what 86.243.228.45 is, or it’s a dead backup server or something, I’d suggest you remove this from the domain record.

⤋ Read More
In-reply-to » Australia To Ban Under-16s From Social Media After Passing Landmark Law Australia will ban children under 16 from using social media after its senate approved what will become a world-first law. From a report: Children will be blocked from using platforms including TikTok, Instagram, Snapchat and Facebook, a move the Australian government argue is necessary to protect their mental health and wellbeing. ... ⌘ Read more

Forcing social media to open their algorithms: hostile to corporations
Forcing young people to not use social media: hostile to young people, helps prevent them from organizing
You think they did this for the benefit of the young people?

Also this ☝️

⤋ Read More

So let’s recap… We’ve got Putin waging war against Ukraine. Netanyahu waging war against Palestine. Iran getting involved. Kim Jong Un helping Russia and sending soldiers as resources for Putin’s war. And now Trump has won a 2nd term in the US where we’ll see him scrap EU sanctions and fines against US companies violating EU laws and what else? 🤔

What dafuq is wrong with this world?! 🌍

⤋ Read More
In-reply-to » @codebuzz I have some shell scripts that handle some of the log formatting details, but I mostly write my mesages by hand. Lately I've been browsing twtxt.net since they aggregate most of the known network. I have a couple of demo aggregators sitting around, but I'm in the middle of some infra rebuilds so a lot of my services are offline rn. They're both built on a simple social graph analysis that extracts urls for your direct follows the follows listed on each of those feeds (friend-of-a-friend replication). certain formatting operations are awkward with my setup, so I may write an app of some kind in the future. likely gemini-based, but I have a number of projects ahead of that one in the queue.

@cuaxolotl@sunshinegardens.org This is largely by accident and not on purpose:

Lately I’ve been browsing twtxt.net since they aggregate most of the known network

⤋ Read More

After the behaviour of a clearly very angry feed author over the past few days, I’m very tempted to give up on Twtxt and allow it to go back to being dead. What really is the point of building and supporting a way to exchange little pieces of text with one another in a completely decentralized way, if you’re just going to keep bumping up against such hostility? I don’t know why I do this anymore.

⤋ Read More

Ya know; Rather than being an asshole and getting all angry, just be reasonable and reach out to the community or folks fetching (or trying) your feed.

Most clients respect caching if your feed is transported I’ve HTTP.

Otherwise you can add the # refresh hint to clients on your feed.

No need to be an obnoxious ass and flood your own feed. That will just get you permanarely unfollowed and ignored.

⤋ Read More
In-reply-to » Yeah.. it is very similar to salty.im a smp is a relay queue for messages. You can self host one if you choose. They also have something called xftp for data storage and device state transfer. You can also self host one.

@doesnm@doesnm.p.psf.lt salty.im needs a lot more work 🤞it is however designed to be 1000% decentralized 👌

⤋ Read More
In-reply-to » should i delete gemini support from twet? iirc in twtxt v2 it starts prohibited. And all of my fields are https

@doesnm@doesnm.p.psf.lt No.

iirc in twtxt v2 it starts prohibited

This is not true. There are no issues supporting fetching feeds via Gemini/Gopher. This is totally fine. What will likely happen is “recommendations” and “drawbacks of using Gemini/Gopher”

⤋ Read More
In-reply-to » I'm looking to develop a static site for twtxt.dev -- A domain I own and have wanted to use for developer and specification docs for Twtxt.

@doesnm@doesnm.p.psf.lt Thanks! I’ve almost come up with my own theme already 🤣 I actually don’t really want to use Hugo at all, I find it too complicated. But it is pretty popular so I thought maybe I’d rip-off a nice theme… Hmmm 🧐

Anyway, What I really normally use for a lot of my static sites is zs

⤋ Read More

I’m looking to develop a static site for twtxt.dev – A domain I own and have wanted to use for developer and specification docs for Twtxt.

Can anyone recommend a few Hugo themes you like?

All of the dev.twtxt.net content would move over as well.

⤋ Read More

👋 Thanks for joining us on our Sept monthly Yarn.social meetup today y’all 🙇‍♂️ We had @david@collantes.us @sorenpeter@darch.dk @doesnm@doesnm.p.psf.lt @falsifian@www.falsifian.org and @xuu@txt.sour.is 💪 Nice turn out! (not all at once of course, as we normally run this over 4 hours as we span many time zones!)

Things we talked about:

  • Decentralised vs. Distributed
  • Use of SHA256 for Twt Hash(es)
  • We solved Edits! 🥳
  • UUID(s) probably won’t work! (susceptible to sppofing)
  • Helped @sorenpeter@darch.dk write some PHP to process/parse User-Agent and service his feed via a custom PHP script 😅
  • @falsifian@www.falsifian.org introduced himself 👌
  • Talked about Merkle Trees 🌳

Did I miss anything? 🤔

⤋ Read More
In-reply-to » Something @anth said on ITC

We:

  • Drop # url= from the spec.
  • We don’t adopt # uuid = – Something @anth@a.9srv.net also mentioned (see below)

We instead use the @nick@domain to identify your feed in the first place and use that as the identify when calculating Twt hashes <id> + <timestamp> + <content>. Now in an ideal world I also agree, use WebFinger for this and expect that for the most part you’ll be doing a WebFinger lookup of @user@domain to fetch someone’s feed in the first place.

The only problem with WebFinger is should this be mandated or a recommendation?

⤋ Read More

Something @anth@a.9srv.net said on ITC

17:42 I should also note in there that it doesn’t address the two things i really want it to: mandate utf-8 (which should be easy to fit in) and something for better @ mentions.

I actually agree with in both counts and it got me thinking…

⤋ Read More

Sharing the comments of the poll (anonymous so I have no idea whom the comments are from):

your poll should include questions about markdown. personally i think inline bits like style, links, images are yes. block quotes, code blocks, bullet lists are mid. but tables and footnotes are no.

Yes sorry about this, I wasn’t able to change much after publishing the poll 😅

⤋ Read More

Starting a couple of new projects (geez where do I find the time?!):

HomeTunnel:

HomeTunnel is a self-hosted solution that combines secure tunneling, proxying, and automation to create your own private cloud. Utilizing Wireguard for VPN, Caddy for reverse proxying, and Traefik for service routing, HomeTunnel allows you to securely expose your home network services (such as Gitea, Poste.io, etc.) to the Internet. With seamless automation and on-demand TLS, HomeTunnel gives you the power to manage your own cloud-like environment with the control and privacy of self-hosting.

CraneOps:

craneops is an open-source operator framework, written in Go, that allows self-hosters to automate the deployment and management of infrastructure and applications. Inspired by Kubernetes operators, CraneOps uses declarative YAML Custom Resource Definitions (CRDs) to manage Docker Swarm deployments on Proxmox VE clusters.

⤋ Read More
In-reply-to » Some more arguments for a local-based treading model over a content-based one:

And finally the legibility of feeds when viewing them in their raw form are worsened as you go from a Twt Subject of (#abcdefg12345) to something like (https://twtxt.net/user/prologic/twtxt.txt 2024-09-22T07:51:16Z).

⤋ Read More
In-reply-to » Some more arguments for a local-based treading model over a content-based one:

There is also a ~5x increase cost in memory utilization for any implementations or implementors that use or wish to use in-memory storage (yarnd does for example) and equally a 5x increase in on-disk storage as well. This is based on the Twt Hash going from a 13 bytes (content-addressing) to 63 bytes (on average for location-based addressing). There is roughly a ~20-150% increase in the size of individual feeds as well that needs to be taken into consideration (on the average case).

⤋ Read More
In-reply-to » Some more arguments for a local-based treading model over a content-based one:

So really your argument is just that switching to a location-based addressing “just makes sense”. Why? Without concrete pros/cons of each approach this isn’t really a strong argument I’m afraid. In fact I probably need to just sit down and detail the properties of both approaches and the pros/cons of both.

I also don’t really buy the argument of simplicity either personally, because I don’t technically see it much more difficult to take a echo -e "<url>\t<timestamp>\t<content>" | sha256sum | base64 as the Twt Subject or concatenating the <url> <timestamp> – The “effort” is the same. If we’re going to argue that SHA256 or cryptographic hashes are “too complicated” then I’m not really sure how to support that argument.

⤋ Read More
In-reply-to » Some more arguments for a local-based treading model over a content-based one:

@sorenpeter@darch.dk Points 2 & 3 aren’t really applicable here in the discussion of the threading model really I’m afraid. WebMentions is completely orthogonal to the discussion. Further, no-one that uses Twtxt really uses WebMentions, whilst yarnd supports the use of WebMentions, it’s very rarely used in practise (if ever) – In fact I should just drop the feature entirely.

The use of WebSub OTOH is far more useful and is used by every single yarnd pod everywhere (no that there’s that many around these days) to subscribe to feed updates in ~near real-time without having the poll constantly.

⤋ Read More
In-reply-to » @lyse I'd suggest making the whole content-type thing a SHOULD, to accommodate people just using some hosting service they don't have much control over. (The same situation could make detecting followers hard, but IMO "please email me if you follow me" is still legit twtxt, even if inconvenient.)

Can someone make the edit?

⤋ Read More

So I whipped up a quick shell script to demonstrate what I mean by the increase in feed size on average as well as the expected increase in storage and retrieval requirements.

$ ./compare.sh
Original file size: 28145 bytes
Modified file size: 70672 bytes
Percentage increase in file size: 151.10%
...

⤋ Read More

👋 Reminder folks of the upcoming Yarn.social monthly online meetup:

I hope to see @david@collantes.us @movq@www.uninformativ.de @lyse@lyse.isobeef.org @xuu@txt.sour.is @sorenpeter@darch.dk and hopefully others too @aelaraji@aelaraji.com @falsifian@www.falsifian.org and anyone else that sees this! 🙏 We’re hopefully going to primarily discuss the future of Twtxt and the last few weeks of discussions 🤣

  • Event: Yarn.social Online Meetup
  • When: 28th September 2024 at 12:00pm UTC (midday)
  • Where: Mills Meet : Yarn.social
  • Cadence: 4th Saturday of every Month

Agenda:

  • Let’s talk about the upcoming changes to the Twtxt spec(s)

#Yarn.social #Meetup

⤋ Read More
In-reply-to » @movq @falsifian @prologic Maybe I don't know what I'm talking about and You've probably already read this: Everything you need to know about the “Right to be forgotten” coming straight out of the EU's GDPR Website itself. It outlines the specific circumstances under which the right to be forgotten applies as well as reasons that trump the one's right to erasure ...etc.

@aelaraji@aelaraji.com This is one of the reasons why yarnd has a couple of settings with some sensible/sane defaults:

I could already imagine a couple of extreme cases where, somewhere, in this peaceful world one’s exercise of freedom of speech could get them in Real trouble (if not danger) if found out, it wouldn’t necessarily have to involve something to do with Law or legal authorities. So, If someone asks, and maybe fearing fearing for… let’s just say ‘Their well being’, would it heart if a pod just purged their content if it’s serving it publicly (maybe relay the info to other pods) and call it a day? It doesn’t have to be about some law/convention somewhere … 🤷 I know! Too extreme, but I’ve seen news of people who’d gone to jail or got their lives ruined for as little as a silly joke. And it doesn’t even have to be about any of this.

There are two settings:

$ ./yarnd --help 2>&1 | grep max-cache
      --max-cache-fetchers int        set maximum numnber of fetchers to use for feed cache updates (default 10)
  -I, --max-cache-items int           maximum cache items (per feed source) of cached twts in memory (default 150)
  -C, --max-cache-ttl duration        maximum cache ttl (time-to-live) of cached twts in memory (default 336h0m0s)

So yarnd pods by default are designed to only keep Twts around publicly visible on either the anonymous Frontpage or Discover View or your Timeline or the feed’s Timeline for up to 2 weeks with a maximum of 150 items, whichever get exceeded first. Any Twts over this are considered “old” and drop off the active cache.

It’s a feature that my old man @off_grid_living@twtxt.net was very strongly in support of, as was I back in the day of yarnd’s design (nothing particularly to do with Twtxt per se) that I’ve to this day stuck by – Even though there are some 😉 that have different views on this 🤣

⤋ Read More
In-reply-to » @falsifian Do you have specifics about the GRPD law about this?

Yeah I’m curious to find out too beyond just “here say”. But regardless of whether we should or shouldn’t care about this or should or shouldn’t comply. We should IMO. I’d have to build something that horrendously violates someone’s rights in another country.

⤋ Read More
In-reply-to » @prologic Do you have a link to some past discussion?

@falsifian@www.falsifian.org Do you have specifics about the GRPD law about this?

Would the GDPR would apply to a one-person client like jenny? I seriously hope not. If someone asks me to delete an email they sent me, I don’t think I have to honour that request, no matter how European they are.

I’m not sure myself now. So let’s find out whether parts of the GDPR actually apply to a truly decentralised system? 🤔

⤋ Read More

Can I get someone like maybe @xuu@txt.sour.is or @abucci@anthony.buc.ci or even @eldersnake@we.loveprivacy.club – If you have some spare time – to test this yarnd PR that upgrades the Bitcask dependency for its internal database to v2? 🙏

VERY IMPORTANT If you do; Please Please Please backup your yarn.db database first! 😅 Heaven knows I don’t want to be responsible for fucking up a production database here or there 🤣

⤋ Read More
In-reply-to » no my fault your client can't handle a little editing ;)

Location Addressing is fine in smaller or single systems. But when you’re talking about large decentralised systems with no single point of control (kind of the point) things like independable variable integrity become quite important.

⤋ Read More

Speaking of AI tech (sorry!); Just came across this really cool tool built by some engineers at Google™ (currently completely free to use without any signup) called NotebookLM 👌 Looks really good for summarizing and talking to document 📃

⤋ Read More

An alternate idea for supporting (properly) Twt Edits is to denoate as such and extend the meaning of a Twt Subject (which would need to be called something better?); For example, let’s say I produced the following Twt:

2024-09-18T23:08:00+10:00	Hllo World

And my feed’s URI is https://example.com/twtxt.txt. The hash for this Twt is therefore 229d24612a2:

$ echo -n "https://example.com/twtxt.txt\n2024-09-18T23:08:00+10:00\nHllo World" | sha1sum | head -c 11
229d24612a2

You wish to correct your mistake, so you make an amendment to that Twt like so:

2024-09-18T23:10:43+10:00	(edit:#229d24612a2) Hello World

Which would then have a new Twt hash value of 026d77e03fa:

$ echo -n "https://example.com/twtxt.txt\n2024-09-18T23:10:43+10:00\nHello World" | sha1sum | head -c 11
026d77e03fa

Clients would then take this edit:#229d24612a2 to mean, this Twt is an edit of 229d24612a2 and should be replaced in the client’s cache, or indicated as such to the user that this is the intended content.

⤋ Read More