@prologic@twtxt.net Oh, wait, there’s already another thread about it. 😅

On second thought, the same rule with the last physically encountered URL when starting parsing from the top applies to prepend-style feeds as well. Much simpler and cleaner this way. Should also fit prepend-style feeds better I reckon.

@falsifian@www.falsifian.org In my opinion it was a mistake that we defined the first url field in the feed to define the URL for hashing. It should have been the last encountered one. Then, assuming append-style feeds, you could override the old URL with a new one from a certain point on:

# url = https://example.com/alias/txtxt.txt
# url = https://example.com/initial/twtxt.txt
<message 1 uses the initial URL>
<message 2 uses the initial URL, too>
# url = https://example.com/new/twtxt.txt
<message 3 uses the new URL>
# url = https://example.com/brand-new/twtxt.txt
<message 4 uses the brand new URL>

In theory, the same could be done for prepend-style feeds. They do exist, I’ve come around them. The parser would just have to calculate the hashes afterwards and not immediately.

@prologic@twtxt.net @bender@twtxt.net That’s exactly the case here with us as well. Maybe not 100% applicable to yarnd, but all other clients that only fetch from their user-controlled subscription list.

@movq@www.uninformativ.de @prologic@twtxt.net Oh yeah, we have to take our time with that and craft it very carefully.

My theory about the descent of the original twtxt universe is that a) people just move on to other things and b) it was just not practical enough.

Nicely put, @movq@www.uninformativ.de! Exactly, reminding people to subscribe etc. is dodgy. To me it feels they think their audience is dumb (and they might be right, I don’t know). Super annoying.

@aelaraji@aelaraji.com Just move to Mars to get an extra hour a day: https://spaceplace.nasa.gov/days/en/ If that’s not enough, Mercury should have you covered for sure.

@prologic@twtxt.net No, it’s all just speculation and I don’t like spreading rumors. 😅 It would be more interesting to hear from the twtxt folks themselves why they stopped working on the original twtxt.

@prologic@twtxt.net

the right way to solve this is to use public/private key(s) where you actually have a public key fingerprint as your feed’s unique identity that never changes

Okay, this is interesting. How would this work in practice? 🤔

@falsifian@www.falsifian.org If the timestamp included a nanosecond part (which is not a valid twtxt feed at the moment, because it mandates RFC3339 timestamps and those only permit one subsecond digit), this could solve the editing problem with little effort. 🤔

Btw, @prologic@twtxt.net, in my experience, people editing their twts is a much more common thing than people changing the URL of their feed. 😅 It breaks threading all the time.

Wow, there are a lot of ideas in this thread already. 😃

@movq@www.uninformativ.de Another idea: just hash the feed url and time, without the message content. And don’t twt more than once per second.

Maybe you could even just use the time, and rely on @-mentions to disambiguate. Not sure how that would work out.

Though I kind of like the idea of twts being immutable. At least, it’s clear which version of a twt you’re replying to (assuming nobody is engineering hash collisions).

In fact, maybe your public key idea is compatible with my last point. Just come up with a url scheme that means “this feed’s primary URL is actually a public key”, and then feed authors can optionally switch to that.

@prologic@twtxt.net Some criticisms and a possible alternative direction:

1. Key rotation. I’m not a security person, but my understanding is that it’s good to be able to give keys an expiry date and replace them with new ones periodically.

2. It makes maintaining a feed more complicated. Now instead of just needing to put a file on a web server (and scan the logs for user agents) I also need to do this. What brought me to twtxt was its radical simplicity.

Instead, maybe we should think about a way to allow old urls to be rotated out? Like, my metadata could somehow say that X used to be my primary URL, but going forward from date D onward my primary url is Y. (Or, if you really want to use public key cryptography, maybe something similar could be used for key rotation there.)

It’s nice that your scheme would add a way to verify the twts you download, but https is supposed to do that anyway. If you don’t trust https to do that (maybe you don’t like relying on root CAs?) then maybe your preferred solution should be reflected by your primary feed url. E.g. if you prefer the security offered by IPFS, then maybe an IPNS url would do the trick. The fact that feed locations are URLs gives some flexibility. (But then rotation is still an issue, if I understand ipns right.)

On the Subject of Feed Identities; I propose the following:

1. Generate a Private/Public ED25519 key pair
   
2. Use this key pair to sign your Twtxt feed
   
3. Use it as your feed’s identity in place of # url = as # key = ...
   

For example:

$ ssh-keygen -f prologic@twtxt.net
$ ssh-keygen -Y sign -n prologic@twtxt.net -f prologic@twtxt.net twtxt.txt

And your feed would looke like:

# nick        = prologic
# key         = SHA256:23OiSfuPC4zT0lVh1Y+XKh+KjP59brhZfxFHIYZkbZs
# sig         = twtxt.txt.sig
# prev        = j6bmlgq twtxt.txt/1
# avatar      = https://twtxt.net/user/prologic/avatar#gdoicerjkh3nynyxnxawwwkearr4qllkoevtwb3req4hojx5z43q
# description = "Problems are Solved by Method" 🇦🇺👨‍💻👨‍🦯🏹♔ 🏓⚯ 👨‍👩‍👧‍👧🛥 -- James Mills (operator of twtxt.net / creator of Yarn.social 🧶)

2024-06-14T18:22:17Z	(#nef6byq) @<bender https://twtxt.net/user/bender/twtxt.txt>  Hehe thanks! 😅 Still gotta sort out some other bugs, but that's tomorrows job 🤞
...

Twt Hash extension would change of course to use a feed’s ED25519 public key fingerprint.

@aelaraji@aelaraji.com My work has this thing called “compressed work”, where you can buy extra time off (as much as 4 additional weeks) per year. It comes out of your pay though, so it’s not exactly a 4-day work week but it could be useful, just haven’t tired it yet as I’m not entirely sure how it’ll affect my net pay

@bender@twtxt.net Yes, they do 🤣 Implicitly, or threading would never work at all 😅 Nor lookups 🤣 They are used as keys. Think of them like a primary key in a database or index. I totally get where you’re coming from, but there are trade-offs with using Message/Thread Ids as opposed to Content Addressing (like we do) and I believe we would just encounter other problems by doing so.

My money is on extending the Twt Subject extension to support more (optional) advanced “subjects”; i.e: indicating you edited a Twt you already published in your feed as @falsifian@www.falsifian.org indicated 👌

Then we have a secondary (bure much rarer) problem of the “identity” of a feed in the first place. Using the URL you fetch the feed from as @lyse@lyse.isobeef.org ’s client tt seems to do or using the # url = metadata field as every other client does (according to the spec) is problematic when you decide to change where you host your feed. In fact the spec says:

Users are advised to not change the first one of their urls. If they move their feed to a new URL, they should add this new URL as a new url field.

See Choosing the Feed URL – This is one of our longest debates and challenges, and I think (_I suspect along with @xuu@txt.sour.is _) that the right way to solve this is to use public/private key(s) where you actually have a public key fingerprint as your feed’s unique identity that never changes.

@aelaraji@aelaraji.com Join the club! 🤣 How about more days in a weekend?! 😱 Bringon #FourDayWorkWeek !!! 🤣

Where do I download more hours for my days? not having more than 24 hours a day S U C K S !

@prologic@twtxt.net do the existing major clients today perform that verification, or is it simply “hey, there is that thingy let’s use it for reference!”?

@bender@twtxt.net Haha, easy to demonstrate. I’ll start an email thread with myself, then you see if you can join in 🤣

@prologic@twtxt.net about this:

“I think Email Message-Id(s) only ever worked because typically you are exchanging emails with recipients you know and vice versa […]

Absolutely not! Maybe this is something best talked. 😅

We can also make use of comments in the feed to build support for detecting/declaring Twts(s) were edited in a feed that are ignored by clients that don’t understand the comments. By design clients ignore comments anyway, but the parser we build for yarnd (which I’d love to turn into a C library that others can just import) can do some interesting things here. @xuu can probably talk more on this…

I think Email Message-Id(s) only ever worked because typically you are exchanging emails with recipients you know and vice versa. It’s much easier to cope with the problems above, because you just ensure your client preserves the Message-Id. Email is a federated system, but by no means is it “decentralised”. You still have to send your email somewhere, not just post it on a website on your own server like Twtxt 😅

There are some subtitles differences like this that makes Message/Thread Id(s) not really that suitable IMO.

@bender@twtxt.net The problem with the approach Email clients do things is;

• How do you come up with the message/thread id in the first place? I’m pretty sure most clients just use a UUID.
  
• How do you know what you’re replying to if you don’t see the message/thread id in the first place?
  
• How do two different users that don’t know each other, but follow the same feed (say /.) make two independent responses forming a thread? What message/thread id do they use? (see above)

@bender@twtxt.net Sorry, trust was the wrong word. Trust as in, you do not have to check with anything or anyone that the hash is valid. You can verify the hash is valid by recomputing the hash from the content of what it points to, etc.

@prologic@twtxt.net I kind of want to think of twtxts as chalk text written on a board hanging in front of the user’s house. As the user is in full control of their own board, they can erase, and rewrite it as they deem fit.

So, how to reply, and engage with something that can potentially change? I think the email based message-id, and in-reply-to would work best, without the rigid boundary of a hash that’s bound to break on edit.

@prologic@twtxt.net you wrote:

“[…] they can trust that the hash is a cryptographic representing of the thread they’re replying to, no matter what.”

It isn’t about trust, is it? There has to be some kind of check to verify the hash is valid, no?

@falsifian@www.falsifian.org Yes;

I don’t think twtxt hashes are long enough to prevent spoofing.

The current spec needs to be updated to expand the hash length to 11 characters to avoid hash collisions (which will happen at some point with 7, if not already).

The issue isn’t dealing with “spoofing”, it’s about solving how clients in a decentralised model agree on the threading model and identity of a thread. Message ID(s) suffer from the fact that as @movq@www.uninformativ.de points out, clients have to “obey” this unwritten rule, but they’re otherwise just arbitrary. Whereas Twt Hashes (I didn’t come up with the idea originally, some smart fellow in cryptography did) are content addressable, meaning that clients don’t have to agree on anything, they can trust that the hash is a cryptographic representing of the thread they’re replying to, no matter what.

@falsifian@www.falsifian.org I like this idea actually for edits.

@movq@www.uninformativ.de @prologic@twtxt.net Another option would be: when you edit a twt, prefix the new one with (#[old hash]) and some indication that it’s an edited version of the original tweet with that hash. E.g. if the hash used to be abcd123, the new version should start “(#abcd123) (redit)”.

What I like about this is that clients that don’t know this convention will still stick it in the same thread. And I feel it’s in the spirit of the old pre-hash (subject) convention, though that’s before my time.

I guess it may not work when the edited twt itself is a reply, and there are replies to it. Maybe that could be solved by letting twts have more than one (subject) prefix.

But the great thing about the current system is that nobody can spoof message IDs.

I don’t think twtxt hashes are long enough to prevent spoofing.

@movq@www.uninformativ.de Care to share your thoughts here?

I don’t know what happened behind the scenes that killed off twtxt (I have a few guesses, though), but the sad truth is that it’s gone.

@prologic@twtxt.net @lyse@lyse.isobeef.org Yeah, same. They say:

If you post quality content and you’ve developed a loyal audience, you should be able to ask your most passionate followers to support you with a premium subscription.

You already can ask your most passionate followers to support you: You can ask for donations.

I regularly donate to people if their content is great and if they actually ask for donations (many just don’t). The platforms for that already exist, I think. 🤔

I’m not interested in the slightest in stuff that has a paywall. “Subscribe for more content!” No, why, go away. Pages that do this immediately feel shady and not trust-worthy. 🤔

Agreed

But I forgot to update the mentions. Oh well. I don’t bother breaking hashes once more, so I just leave it at that. :-)

@prologic@twtxt.net I simply have absolutely no interest in selling my thoughts to others. I don’t like things behind paywalls. Each to their own, but I’m not gonna contribute to that.

(Let’s not rush things, obviously. Such a change would have to be well thought through and actually be worth it. It’s not like the current state of Yarn/twtxt is completely unusable.)

@lyse@lyse.isobeef.org @prologic@twtxt.net Sorry, I have hardly slept last night. 😅 I probably didn’t chose the best words to describe this. 🥴

Yes, I’m all for dedicated message IDs. That would be a whole new format then. But I would be fine with it.

Honestly, me too. When Yarn originally showed up, I was concerned that it would extend twtxt in dramatically incompatible ways or, worse, change it in a way so that you needed server software. 😅 The latter would have ruined it for me. A major reason why I still use twtxt/Yarn is that it’s still just a file you put somewhere. If there was the need to run a daemon, I’d give up and just use some ActivityPub thingy instead.

What I did not expect, however, was that the original twtxt itself would just … die. There has been no development in the original software anymore and virtually all the original feeds are dead. Some feeds are left, but they’re just used as an alternative to Atom/RSS for some blogs. I don’t know what happened behind the scenes that killed off twtxt (I have a few guesses, though), but the sad truth is that it’s gone.

So, yeah, maybe this gives us the freedom now to break with the original twtxt spec (if needed) and come up with a format that fixes the issues we’re seeing.

(Oh god. Are we re-inventing Usenet then? Again? 😂)

@prologic@twtxt.net https://lyse.isobeef.org/tmp/hashes.png Yep, broken hashes. I just fixed them after refollowing on the new URL (my client doesn’t know metadata fields).

@prologic@twtxt.net Yes, that’s when I noticed. Luckily, I had another knive in my bagpack. But I cannot find the folding one at home either. Damn. I already lost my other knive at the flea market in May this year, now the next one. This sucks.

@bender@twtxt.net A Fred changed its url metadata field 🤣

@prologic@twtxt.net “I’m now 100% certain this is what happened”, what exactly happened?

After unfollowing and refollowing on the new feed URL, I’m now 100% certain this is what happened for @cuaxolotl@sunshinegardens.org 🤣 The real problem is really this:

How do we identify a feed?

It cannot be the URL, because the author could change where they serve it from. This was as “good” as we could get it, but time and time again this has proven to be problematic for, well, a few folks that change their mind, which frankly should be allowed 😅

For supporting edits, I was thinking more along the lines of: If a client edits a Twt already published, it should put the hash of the previous Twt. Something like:

2024-09-05T13:37:40+00:00   (edit:mp6ox4a) Hello world!

To be honest, I don’t really see “editing” as a problem. I see that as a natural behavior of “forking” in the first place, that just forms a. new sub-tree. What’s really problematic here is when a feed author changes the “identity” of their feed and changes the # url = metadata field, which is what I believe @cuaxolotl@sunshinegardens.org has just done, though I’m not 100% certain, I’m like 98% sure haha 😝

@lyse@lyse.isobeef.org Did you check your pocket? 🤣

@movq@www.uninformativ.de It sounds complicated. After reading it only twice, I haven’t gotten it. :-D

Yes, I’m all for dedicated message IDs. That would be a whole new format then. But I would be fine with it. The only thing is that all our clients have to be touched. At the moment, I do not worry about spoofing (however, I definitely should).

Fuck, I lost my pocket knive somewhere.

@movq@www.uninformativ.de Sorry but what’s a partial hash exactly? 🤔

All this hash breakage made me wonder if we should try to introduce “message IDs” after all. 😅

But the great thing about the current system is that nobody can spoof message IDs. 🤔 When you think about it, message IDs in e-mails only work because (almost) everybody plays fair. Nothing stops me from using the same Message-ID header in each and every mail, that would break e-mail threading all the time.

In Yarn, twt hashes are derived from twt content and feed metadata. That is pretty elegant and I’d hate see us lose that property.

If we wanted to allow editing twts, we could do something like this:

2024-09-05T13:37:40+00:00   (~mp6ox4a) Hello world!

Here, mp6ox4a would be a “partial hash”: To get the actual hash of this twt, you’d concatenate the feed’s URL and mp6ox4a and get, say, hlnw5ha. (Pretty similar to the current system.) When people reply to this twt, they would have to do this:

2024-09-05T14:57:14+00:00	(~bpt74ka) (<a href="https://txt.sour.is/search?q=tags:hlnw5ha">#hlnw5ha</a>) Yes, hello!

That second twt has a partial hash of bpt74ka and is a reply to the full hash hlnw5ha. The author of the “Hello world!” twt could then edit their twt and change it to 2024-09-05T13:37:40+00:00 (~mp6ox4a) Hello friends! or whatever. Threading wouldn’t break.

Would this be worth it? It’s certainly not backwards-compatible. 😂