It feels like an A’ Hole pointing at typos while other people are the ones doing the real work ! 😅

@bender@twtxt.net Now that I’m thinking about it, I could just add in a cron job on my remote machine with: twtxt2html https://domain.ltd/twtxt.txt > /path/to/static_files_dir/ that way I could benefit from the ‘relative time’ portion I’m getting rid of with the -n …

@aelaraji@aelaraji.com I just added support for passing a custom template file via -T/--template in case you need a custom template 👌

prologic@JamessMacStudio
Wed Sep 18 01:27:29
~/Projects/yarnsocial/twtxt2html
 (main) 130
$ ./twtxt2html --help
Usage: twtxt2html [options] FILE|URL

twtxt2html converts a twtxt feed to a static HTML page
  -d, --debug             enable debug logging
  -l, --limit int         limit number ot twts (default all) (default -1)
  -n, --noreldate         do now show twt relative dates
  -r, --reverse           reverse the order of twts (oldest first)
  -T, --template string   path to template file
  -t, --title string      title of generated page (default "Twtxt Feed")
  -v, --version           display version information
pflag: help requested

@prologic@twtxt.net I’d be glad to! I’m just taking time to get well acquainted with it’s ins and out before saying something stupid 😅 Like… I’ve just noticed the -n 🫠

hehe, a cara da sra do café quando uma turista espanhola com alergia ao glúten perguntou se podia fazer uma francesinha sem pão

@movq@www.uninformativ.de Oh! 🤣 For some reason I thought my first feed was lost and gone 🤣 Hahahaha it’s been there the whole time 😅 Fuck we write good specs and software 🥳

@falsifian@www.falsifian.org I like this idea too as a completing solution to the “identify problem”:

Or maybe url changes could somehow be combined with the archive feeds extension? Could the url metadata field be local to each archive file, so that to switch to a new url all you need to do is archive everything you’ve got and start a new file at the new url?

👌

One thing that’s on my mind over the last few days about all this Twt editing and identity stuff we’ve been having hot debates over is this…

I don’t really have a problem with editing twts, or someone changing their feed’s URL.

Personally I think the folks that do are rightfully pedantic and like a good user experience, which I don’t blame ‘em. I would expect the same too. Anyway, just wanted to get that out there, I believe we can support editing and identity in a way that is still simple, as long as we bring clients along for the ride with us. The old/legacy original client though will have to remain well, ya know 😅

@prologic@twtxt.net

(#w4chkna) @falsifian@www.falsifian.org You mean the idea of being able to inline # url = changes in your feed?

Yes, that one. But @lyse@lyse.isobeef.org pointed out suffers a compatibility issue, since currently the first listed url is used for hashing, not the last. Unless your feed is in reverse chronological order. Heh, I guess another metadata field could indicate which version to use.

Or maybe url changes could somehow be combined with the archive feeds extension? Could the url metadata field be local to each archive file, so that to switch to a new url all you need to do is archive everything you’ve got and start a new file at the new url?

I don’t think it’s that likely my feed url will change.

@movq@www.uninformativ.de Well at this point I think I’m going to try to combine @lyse@lyse.isobeef.org’s idea for supporting moving your feed to a different URL and this idea for supporting editing. I’ll spec it up and see if what we think from there…

@aelaraji@aelaraji.com Btw, I’m also open to ideas for this tool and welcome any contributions 👌

@bender@twtxt.net It’s just a simple twtxt2html and scp … it goes like:

twtxt2html $HOME/path/to/local_twtxt_dir/twtxt.txt > $HOME/path/to/local_twtxt_dir/log.html && \
scp $HOME/path/to/local_twtxt_dir/log.html user@remotehost:/path/to/static_files_dir/

I’ve been lazy to add it to my publish_command script, now I can just copy/pasta from the twt 😅

@movq@www.uninformativ.de I did the same. jenny fetches archives, yes, but that twtxt I am referring about is no longer. If you fetch it, but I don’t, there is certainly something going on…

It would mean clients that support the TwtSubject and TwtHash extension, should also indicate the previous version of their Twt when editing.

Ultimately I think we just need to agree on a way to represent an edit and the previous version of a Twt in a way that makes sense. I like one of the ideas presented earlier in some other thread (god only knows which one haha 😝); That is: <timestamp> (#hash;#originalHash) <content> For example.

@movq@www.uninformativ.de I think if Git can solve the same problem of branching, forking, patching and merging, so can we 🤣

@movq@www.uninformativ.de No that’s okay. I happen to agree with you really, I just wanted to get a bit of a vibe on using cryptography in general and the idea of signing feeds. It’s not particularly about a problem being solved, just gauging your opinions/thoughts on this 👌

@movq@www.uninformativ.de I did started from scratch, today. I using am commit 6e8ce5afdabd5eac22eae4275407b3bd2a167daf (HEAD -> main, origin/main, origin/HEAD), I keep myself up-to-date, LOL. Still, that specific twtxt (o6dsrga) is no longer.

@aelaraji@aelaraji.com We just have to write better clients 🤣 I have figured out how to detect edits FWIW, but haven’t gone from R&D phase to an actual design and implementation. But it is possible to detect an edit as well as a similarity score and the matching Twts.

@sorenpeter@darch.dk It’s nobody’s fault! 😇 It’s all part of the fun with them Ones and Zeros

@quark@ferengi.one I think I may have lost my original feed because it was written with the old software before Yarn social became a thing.

@sorenpeter@darch.dk not really you’re really forming a cryptographic chain of twts, that are cryptographically provable by anyone, at least in one direction ). It’s called content addressing. Your propose scheme while simple doesn’t do this.

After re-fetching feeds, the earliest twtxt I have from you is n7gavoa.

@prologic@twtxt.net, your first twtxt ever was o6dsrga, but I can’t find the source for it (the raw file). I reset everything, and re-fetched fresh feeds (allegedly including archives). Where is it?

@movq@www.uninformativ.de I figured it will be something like this, yet, you were able to reply just fine, and I wasn’t. Looking at your twtxt.txt I see this line:

2024-09-16T17:37:14+00:00	(#o6dsrga) @<prologic https://twtxt.net/user/prologic/twtxt.txt>

@<quark https://ferengi.one/twtxt.txt> This is what I get. 🤔

Which is using the right hash. Mine, on the other hand, when I replied to the original, old style message (Message-Id: <o6dsrga>), looks like this:

2024-09-16T16:42:27+00:00	(#o) @<prologic https://twtxt.net/user/prologic/twtxt.txt> this was your first twtxt. Cool! :-P

What did you do to make yours work? I simply went to the oldest @prologic@twtxt.net’s entry on my Maildir, and replied to it (jenny set the reply-to hash to #o, even though the Message-Id is o6dsrga). Since jenny can’t fetch archived twtxts, how could I go to re-fetch everything? And, most importantly, would re-fetching fix the Message-Id:?

@prologic@twtxt.net you will always be replying to OP - that is what the twthash is a shorthand for, it it not?!

no my fault your client can’t handle a little editing ;)

@movq@www.uninformativ.de I’m glad you like it. A mention (@<movq https://www.uninformativ.de/twtxt.txt>) is also long, but we live with it anyway. In a way a replyto: is just a mention of a twt instead of a feed/person. Maybe we chould even model the syntax for replies on mentions: (#<2024-09-17T08:39:18Z https://www.eksempel.dk/twtxt.txt>) ?!

@lyse@lyse.isobeef.org and @movq@www.uninformativ.de and possibly @aelaraji@aelaraji.com and even @cuaxolotl – I’m very curious to understand and hear thoughts, pros and cons or other feelings about introducing the notation of a feed’s identify using cryptography? If we were to keep things simple, and use what’s commonly available, for example SSH ED25519 keys? using the ssh-keygen -Y sign or ssh-keygen -Y verify tools already available? Maybe in combination with @xuu@txt.sour.is ’s idea of generating a random unique ID for your feed, say # id = and signing it with your ED25519 key? 🔑

@falsifian@www.falsifian.org You mean the idea of being able to inline # url = changes in your feed?

Whatever gets used, it would be nice to be able to rotate identities. I like @lyse@lyse.isobeef.org’s idea for that.

GENESYSM-ADN6: A 3.5″ SubCompact System with Triple 2.5GbE LAN Ports
GENESYSM-ADN6: A 3.5” SubCompact System with Triple 2.5GbE LAN Ports ⌘ Read more

@mckinley@twtxt.net Yes, changing domains is be a problem if you tie your identity to an https url. But I also worry about being stuck with a key I can’t rotate. Whatever gets used, it would be nice to be able to rotate identities. I like @lyse@lyse.isobeef.org’s idea for that.

This scheme also only support threading off a specific Twt of someone’s feed. What if you’re not replying to anyone in particular?

Noting that this scheme cannot support disjoint threads that should be merged together once either party discovers each other 😅

@movq@www.uninformativ.de It does now as of several weeks ago or so 👌

@lyse@lyse.isobeef.org Yea I think your idea of inclining url changes in the feed works perfectly as long as folks remember to do so I guess? 🤔

@bender@twtxt.net Seems to have used the hash correctly here 🤣

Holy shot! what an old thread 🤣

@cuaxolotl@sunshinegardens.org Gitea is plenty fast for me 👌 Even with a SQLite database 🤣

I dunno whether any of this is actually true 🤷‍♂️ The LLM 🤖could have made (hallucinated) this shit up 🤣

WiscKey’s approach to handling key-value pairs in SSDs offers several advantages:

1. It minimizes I/O amplification by separating keys and values, allowing for more efficient storage and retrieval.
   
2. The design leverages the SSD’s performance characteristics, utilizing sequential writes and parallel random reads to enhance throughput and reduce latency.
   
3. WiscKey maintains excellent insert and lookup performance while improving SSD lifespan by reducing the number of erase cycles required.

WiscKey minimizes CPU usage compared to LevelDB by eliminating the log file, which reduces the CPU cost associated with encoding and writing key-value pairs. While LevelDB has higher CPU usage due to its single writer protocol and background compaction using one thread, WiscKey’s architecture allows for garbage collection to be decoupled and performed later, minimizing its impact on foreground performance. Consequently, WiscKey generally exhibits lower CPU usage during workloads, except when utilizing multiple background threads for prefetching.

The four critical ideas in the design of WiscKey are:

1. Separation of keys from values, with only keys stored in the LSM-tree and values in a separate log file.
   
2. Utilization of the parallel random-read characteristic of SSD devices to handle unsorted values during range queries.
   
3. Implementation of unique crash-consistency techniques to manage the value log efficiently.
   
4. Optimization of performance by removing the LSM-tree log without sacrificing consistency, reducing system-call overhead from small writes.

Summary of WiscKey: Separating Keys from Values in SSD-Conscious Storage

• Authors: Lanyue Lu, Thanumalayan Sankaranarayana Pillai, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau
  
• Conference: 14th USENIX Conference on File and Storage Technologies (FAST ‘16)
  
• Key Concept: WiscKey is a key-value store that separates keys from values to minimize I/O amplification, optimizing performance for SSDs.
  
• Performance: WiscKey outperforms LevelDB and RocksDB in various workloads, achieving up to 111× faster loading and improved random lookup speeds.
  
• Design Goals: Focus on low write/read amplification, SSD optimization, and support for modern features like range queries.

afaik nobody has done this, but i really need some numbers that can indicate the relative performance of various git servers (cgit, gitea, gitlab) on comparable hardware. cgit claims to be hyperfast, but what does that mean in practice?

afaik nobody has done this, but i really need some numbers that can indicate the relative performance of various git servers (cgit, gitea, gitlab) on comparable hardware. cgit claims to be hyperfast, but what does that mean in practice?

Can anyone recommend a decent Android ROM that strips out as much of the spyware as possible? Is GrapheneOS a good option? I need to get a new phone anyway so I don’t mind buying within a supported device list as long as I can get one on the used market for $300-$400 or less.

If anyone could recommend some learning resources for this stuff I’d really appreciate it.

Thank you @movq@www.uninformativ.de Things are working again!! 🙏

@movq@www.uninformativ.de I’d guess the same goes for all twtxt.social feeds… I can’t see bender’s archived twts either, didn’t check for the others.

This is how my original message shows up on jenny:

From: quark <quark>
Subject: (#o) @prologic this was your first twtxt. Cool! :-P
Date: Mon, 16 Sep 2024 12:42:27 -0400
Message-Id: <k7imvia@twtxt>
X-twtxt-feed-url: https://ferengi.one/twtxt.txt

(#o) @<prologic https://twtxt.net/user/prologic/twtxt.txt> this was your first twtxt. Cool! :-P

Hmm… I replied to this message:

From: prologic <prologic>
Subject: Hello World! 😊
Date: Sat, 18 Jul 2020 08:39:52 -0400
Message-Id: <o6dsrga>
X-twtxt-feed-url: https://twtxt.net/user/prologic/twtxt.txt

Hello World! 😊

And see how the hash shows… Is it because that hash isn’t longer used?

@prologic@twtxt.net this was your first twtxt. Cool! :-P

@movq@www.uninformativ.de I wiped both ~/.cache/jenny and my maildir_target when I tried to reset things. Still got wrecked 😅

If it’s not too much to ask, could you backup or/change your maildir_target and give it a try with an empty directory?

@prologic@twtxt.net

PS: I still can’t get your and bender’s archived twts (at least the ones I’ve noticed), nor can I --fetch-context on replays to them. your oldest is the one from 2024-06-14 18:22 … I can see lyse’s tho! but I doubt this is related the edit issue but this helps with something.

@prologic@twtxt.net I can’t pinpoint the exact cause but here are a couple of symptoms I observed:

• It all started with a LOT of his old twts starting back in 2020 showing in a weird way, some were empty others were duplicates and a lot more got marked for deletion by neomutt with the D tag.
  
• After trying to restart things with a fresh Maildir, I couldn’t fetch a lot of twts, even mine which was a replay to one of his. but then I was able to after temporarily deleting his link from my follow file.
  

then @quark@ferengi.one and @bender@twtxt.net pointed out the inconsistent from: + feed url and the twt edit

@movq@www.uninformativ.de we can shorten it by six characters, with (r:https://...). 😅

@quark@ferengi.one Meh I lost the plot ages ago 🤣

@quark@ferengi.one Haha okay 😅

@prologic@twtxt.net I am going to light some candles this weekend to “La Virgen de Macarena” to make it happen! :-D

@prologic@twtxt.net you need to catch up with my twtxts, mate. :-P

@movq@www.uninformativ.de Bah you’re right, that’s a mistake and easily fixed 😅

@movq@www.uninformativ.de I tend to agree too, I think the focus should be on fixing and supporting Edits first 👌

@quark@ferengi.one We will fix this soon™ 🔜

@aelaraji@aelaraji.com So what is it about @sorenpeter@darch.dk’s feed that’s screwed with your client? (Jenny?) 🤔 Kind of curious now 🤣

@aelaraji@aelaraji.com Yes, according to the spec we wrote for Archived Extension:

The second value of prev is a name relative to the base directory of the feed’s URL in url (more specifically, in the URL that the client used to retrieve the feed). In the example above, prev would evaluate to the full URL https://example.com/twtxt-2021-10-18.txt for HTTPS and gopher://example.com/0/twtxt-2021-10-18.txt for Gopher.

@prologic@twtxt.net by the way and just in case… is the metadata in tour twtxt.txt file, pointing at your rotated feed files formatted as prev = hash twtxt.txt/n instead of a link by design? I couldn’t fetch any, nor can I do a –fetch-context on replays to your old twts.

@aelaraji@aelaraji.com grats! See how much trouble an edited twtxt can cause? Wish there was a simpler solution. Alas, I don’t have much hope.

Done and done! everything is back to normal! 🥳

@aelaraji@aelaraji.com LOL 😂

FIX: Temporarily removed sorenpeter’s twtxt link from my follow list, whipped my twtxt Maildir and jenny Cache. Only then I was able to fetch everything as usual (I think). Now I’ll backup things and see what happens if I pull sorenpeter’s feed.

No keyboards were harmed during this experiment… yet.

On my blog: Developer Diary, Ozone Day https://john.colagioia.net/blog/2024/09/16/ozone-layer.html #programming #project #devjournal

@quark@ferengi.one It would also be possible to use the search engine here too I think 🤔 i.e: https://search.twtxt.net

@quark@ferengi.one Looks like that would work according to the patch I just read 👌

These then become useful in filters like what you see here:

It’s useful to know however that such feeds are actually marked as type=rss (e.g: https://feeds.twtxt.net/slashdot/twtxt.txt), just as feeds like @tiktok@feeds.twtxt.net are marked as type=bot

@aelaraji@aelaraji.com Ahh that’s interesting! 🧐 One of my original goals when I started out building Yarn.social was to also be a source of news, blogs, and whatever else that could be roughly/easily translated into a Twtxt feed. I’m not sure if others do something similar, but that’s why I built feeds.twtxt.net, which consumes RSS/Atom and produces Twtxt feeds.

My only desire one day is to build a “Feed Builder” of sorts that allows one to say, for example, construct a Slashdot feed but without AI hype, or as another example, a BBC/ABC feed that’s a digest once or twice per day.

@prologic@twtxt.net Nah! I don’t do news feeds 🤣 I gave some a try back then but it was just way too much noise. I have a separate app for RSS feeds I want to follow. None of them mention AI except for one article about the author’s fight back against the crawlers, I believe I’ve mentioned it before.

@bender@twtxt.net Ack 👌

@aelaraji@aelaraji.com Good man 🤣 I keep getting this bloody AI hype from various news feeds I subscribe to via Twtxt like Slashdot cough 🤦‍♂️

The wiered thing is Twtxt fetches everything just fine (I think) except for not having the convenience of having replays grouped into threads.

@movq@www.uninformativ.de I can have more than one Yarn, correct? Like:

"yarn_pods_for_discovery": ["https://twtxt.net", "https://txt.sour.is"],

I mean, this: https://darch.dk/timeline/replies?url=http://darch.dk/twtxt.txt

@aelaraji@aelaraji.com make sense, probably. The twtxt was already on my Maildir, that’s why I can fetch it. I fetch every 3 minutes (sssh, don’t tell anyone!). LOL!

@aelaraji@aelaraji.com check “Replies”. :-D

@bender@twtxt.net I can’t see ANY of those LOL not even a broken thread. The whole Thread went Poof!! as if it has never happened …

Bonus: On his Pod/Profile it shows as if his last twt is from 4 Months ago.

Spoiler: Didn’t work. LOL

@quark@ferengi.one No can do! I can’t see any of the replies to that thread, not even mine LOL. let me se if I can fetch @sorenpeter@darch.dk ’s feed with the https link.

More:

Subject: The [tag URI scheme](https://en.wikipedia.org/wiki/Tag_URI_scheme) looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be
        somewhat trivial to parse. But there are still the issue with what the name/id should be... Maybe it doesn't have to bee that stick? Instead of using `tag:` as the prefix/protocol, it would more it clear
        what we are talking about by using `in-reply-to:` (https://indieweb.org/in-reply-to) or `replyto:` similar to `mailto:` 1. `(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)' 2.
        `(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)' 2. `(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)' I know it's longer that 7-11 characters, but it's self-explaining when looking at the
        twtxt.txt in the raw, and the cases above can all be caught with this regex: `\([\w-]*reply[\w-]*\:` Is this something that would work?

Subject: The [tag URI scheme](https://en.wikipedia.org/wiki/Tag_URI_scheme) looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be
        somewhat trivial to parse. But there are still the issue with what the name/id should be... Maybe it doesn't have to bee that stick? Instead of using `tag:` as the prefix/protocol, it would more it clear
        what we are talking about by using `in-reply-to:` (https://indieweb.org/in-reply-to) or `replyto:` similar to `mailto:` 1. `(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)` 2.
        `(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)` 3. `(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)` I know it's longer that 7-11 characters, but it's self-explaining when looking at the
        twtxt.txt in the raw, and the cases above can all be caught with this regex: `\([\w-]*reply[\w-]*\:` Is this something that would work?

Notice the difference? Soren edited, and broke everything.

Two different “from” too:

"sorenpeter (soren)" <sorenpeter>
sorenpeter <sorenpeter>

See:

Message-Id: <hns535a@twtxt>
X-twtxt-feed-url: https://darch.dk/twtxt.txt
In-Reply-To: <pvju5cq@twtxt>

And

Message-Id: <weadxga@twtxt>
X-twtxt-feed-url: http://darch.dk/twtxt.txt
In-Reply-To: <pvju5cq@twtxt>

Two feed URLs, one HTTPS, the other HTTP.

@aelaraji@aelaraji.com no, it is not just you. Do fetch the parent with jenny, and you will see there are two messages with different hash. Soren did something funky, for sure.