Woot, thank you! Using a config.json like this:

{
  "host": "localhost:31212",
  "protocols": ["http"]
}

Indeed did the trick! I know it isn’t production ready, but I wanted to see with my own eyes, locally, how did it look. :-) I like where you are going! It is looking very nice, and polished. Can’t wait for an alpha, beta, and release!

@dce@hashnix.club No worries 😌 It’s all documented in our soecs, it’s not such a common thing that we’ve felt the great need to really solve, we’re aware folks want to sometimes have their feed on several protocols, and that’s totally fine™ 😅

@dce@hashnix.club Ah, oh, well then. 🥴

My client supports that, if you set multiple url = fields in your feed’s metadata (the top-most one must be the “main” URL, that one is used for hashing).

But yeah, multi-protocol feeds can be problematic and some have considered it a mistake to support them. 🤔

@lyse@lyse.isobeef.org They are optional dependencies and listed as such:

$ pacman -Qi pinentry
Name            : pinentry
Version         : 1.3.1-5
Description     : Collection of simple PIN or passphrase entry dialogs which
                  utilize the Assuan protocol
Optional Deps   : gcr: GNOME backend [installed]
                  gtk3: GTK backend [installed]
                  qt5-x11extras: Qt5 backend [installed]
                  kwayland5: Qt5 backend
                  kguiaddons: Qt6 backend
                  kwindowsystem: Qt6 backend

And it’s probably a good thing that they’re optional. I wouldn’t want to have all that installed all the time.

@movq@www.uninformativ.de Yeah, you gotta love when a “totally real decentralized protocol” does that. How are they doing this and still pretending to be one, is beyond me.

The WM_CLASS Property is used on X11 to assign rules to certain windows, e.g. “this is a GIMP window, it should appear on workspace number 16.” It consists of two fields, name and class.

Wayland (or rather, the XDG shell protocol – core Wayland knows nothing about this) only has a single field called app_id.

When you run X11 programs under Wayland, you use XWayland, which is baked into most compositors. Then you have to deal with all three fields.

Some compositors map name to app_id, others map class to app_id, and even others directly expose the original name and class.

Apparently, there is no consensus.

@bender@twtxt.net Yeah, well, it’s a bit like twtxt. There is a Gopher community, but it’s small. I actually don’t like that HTTP is so easily accessible. I don’t like it that much when people post links to my site on HackerNews or something like that. Too much exposure.

Gopher is a small world. It’s slow and cozy.

And much like twtxt, the protocol is simple®, so it’s easier to tinker with it.

Gopher server is back online and I’ll be phasing out Mastodon.

gopher://uninformativ.de

(No, I won’t do multi-protocol twtxt again. 😅)

When I chose the MIT license for all of my software, I thought:

“Should I use GPL, which I don’t really understand? Is that worth it? Yeah, there is a theoretical possibility that some company might use my code in their proprietary product … and then what? Should I sue them to enforce the GPL? I’m not going to do that anyway, so I’ll just use the MIT license.”

And now we have those LLM scrapers and now it’s suddenly a reality that these companies (ab)use my code. I can see it in my logs. I didn’t expect that back then.

GPL wouldn’t help, either, of course. (Regardless, I now think that GPL would have been the better choice anyway.)

I’m honestly considering taking my code and website offline. Maybe make it accessible through some obscure protocol like Gopher or Gemini, but no more HTTP.

(Yes, Anubis might help. Temporarily.)

I’m just tired.

@andros@twtxt.andros.dev Thanks for consolidating a lot of good ideas. Especially how you have deiced to just extend the mention syntax for location-based treads. This might even be backward compatible with older (pre-yarn) clients.
What about using Z for UTC +00:00- is that allowed in your specs?
Regarding url = I would suggest to only allow one and the maybe add url_old = or url_alt = !?
I’m still not a fan of a DM feature, even thou it helps that i have now been split out into a separate feed file. Instead if would suggest a contact = field for where people can put an email or other id/link for an established chat protocol like signal or matrix.

@movq@www.uninformativ.de @kat@yarn.girlonthemoon.xyz @quark@ferengi.one In 2014 one person created protocol ii. Later it forked in IDEC. Why i said this? Because it’s simple “federated” forum-like protocol where from your station fetch another every 5-10 minutes. Stations has topic-based channels like idec.talks, linux.16, haiku.os, zx.spectrum. In short it’s FIDO but.. more modern? Documentation: https://github.com/idec-net/new-docs (mostly Russian, but you can use translator, also protocol already translated to english)

Confession:

I’ve never found microblogging like twtxt or the Fediverse or any other “modern” social media to be truly fulfilling/satisfying.

The reason is that it is focused so much on people. You follow this or that person, everybody spends time making a nice profile page, the posts are all very “ego-centric”. Seriously, it feels like everybody is on an ego-trip all the time (this is much worse on the Fediverse, not so much here on twtxt).

I miss the days of topic-based forums/groups. A Linux forum here, a forum about programming there, another one about a certain game. Stuff like that. That was really great – and it didn’t even suffer from the need to federate.

Sadly, most of these forums are dead now. Especially the nerds spend a lot of time on the Fediverse now and have abandoned forums almost completely.

On Mastodon, you can follow hashtags, which somewhat emulates a topic-based experience. But it’s not that great and the protocol isn’t meant to be used that way (just read the snac2 docs on this issue). And the concept of “likes” has eliminated lots of the actual user interaction. ☹️

that said, and reading to @sorenpeter@darch.dk and @andros@twtxt.andros.dev I have new thoughts. I assume that this won’t change anyone’s opinions or priorities, so it makes no harm sharing them.

It’s always tempting to use something that already exists (like X, Masto, Bsky, etc.) rather that building anything through effort and disagreement until reaching to something useful and valuable together. A ‘social service’ is only useful if people is using it.

I’ll add that I haven’t lost interest on the ‘hacky’ part of twtxt about developing tools, protocols, and extensions as a community. It’s the appealing part! It’s a nice hobby to have, shared with random people across the world.
But this is not the right way for me, and makes me feel that I’m unwelcome to propose something different (after watching replies to my previous twt). Feels like “If you don’t agree, you are free to leave, we’ll miss you.” Naah, not cool. I’ve lived that many times before, and nowadays I don’t have enough spare time and energy for a hobby like that.

Let’s see what happens next with the micro-community!

Nobody writes emails by hand using RFC 5322 anymore, nor do we manually send them through telnet and SMTP commands. The days of crafting emails in raw format and dialing into servers are long gone. Modern email clients and services handle it all seamlessly in the background, making email easier than ever to send and receive—without needing to understand the protocols or formats behind it! #Email #SMTP #RFC #Automation

i feel like i’ve stumbled upon an alternative internet universe, gemini protocol, the small internet and a counter-cultural world!

I’m thinking of building a hardened peering protocol for Yarn.social’s yarnd: pods establish cryptographic identities, exchange signed /info and /twt payloads with signature verification, ensuring authenticity, integrity, and spoof-proof identity validation across the distributed network.

@prologic@twtxt.net OH SHIT using this for a protocol like gopher is smart! might have to try that for gemini so i don’t have to keep a port open for that

@prologic@twtxt.net @aelaraji@aelaraji.com It depends! If you are working with rsync and scp with the same protocol… I want to know! 😁

@aelaraji@aelaraji.com What protocol do you use?

@prologic@twtxt.net twtxt DM is not a serious DM protocol.

Fascinating read on the emerging Model Context Protocol — a new standard for integrating LLMs with agents and tools.

@bender@twtxt.net thinked about Gemini protocol. Why corporations shit this name with cryptocurrency and LLMs?

well, I assume by syntax you mean Gemtext (which I like a lot, my personal blog is built on top of it), so I think it might work for twtxt clients…

I knew of twtxt in Gemini Antenna, so at least the 2017 spec might work on that protocol. I think the main issue with extensions is that they weren’t designed with many URLs and protocols in mind.

Also I have to admit that the Gemini community significantly reduced in the last few years. I don’t know how worth it is to add support for Gemini now.

@eapl.me@eapl.me I agree. The syntax is weird inside Gemini and twtxt is made with the http protocol in mind and Gemini doesn’t work with some extensions.

well… it has been an opportunity to build an artisanal microblogging client on top of a minimalist protocol. I agree on the hacker toy part.

And of course it’s about being part of a niche community which is (mostly) amazing, and nurturing. As there is almost no one writing in my native spanish, it has been an interesting challenge to share my thoughts in english, as well.

I couldn’t say it’s a ‘social network’ per se, I think it lack many engagement things usually associated with social networks, although it has a social part of igniting discussions, learnings and behavioral changes, which is the meaning of social for me.

@prologic@twtxt.net All the URL are missing the protocol part (https://) and my markdown parser does not know how to handle but I see yarnd does it just fine.

@falsifian@www.falsifian.org
it look like your markdown image tags are missing the protocol part (https://) so they don’t render at least on my server: https://darch.dk/timeline/conv/3vtnszq

@doesnm@doesnm.p.psf.lt Threema is also known to be crippled to state actors and the five eyes. It has known crypto protocol weaknesses that can leak metadata.

It’s ok for most encrypted protocols (In salty you can fetch other messages but can’t decrypt). Btw i think recipient can be removed so if someone seen message they tried to decypt, if can’t - its not message to you

Why not nostr way? https://github.com/nostr-protocol/nips/blob/master/05.md#showing-just-the-domain-as-an-identifier

One benefit with bluesky is your username is also a website. And not a clunky URL with slashes and such. I wish twtxt adopted that. I have advocated for webfinger to for twtxt to let us do something like it with usernames. Nostr has something like it

By default the bsky.social urls all redirect to their feeds like: hmpxvt.bsky.social
Many custom urls will redirect to some kind of linktree or just their feed cwebonline.com or la.bonne.petite.sour.is or if you are a major outlet just to your web presence like https://theonion.com‬ or https://netflix.com

Its just good SEO practice

Do all nostr addresses take you to the person if typed into a browser? That is the secret sauce.
No having to go to some random page first. no accounts. no apps to install. just direct to the person.

@eapl.me@eapl.me here are my replies (somewhat similar to Lyse’s and James’)

1. Metadata in twts: Key=value is too complicated for non-hackers and hard to write by hand. So if there is a need then we should just use #NSFS or the alt-text file in markdown image syntax ![NSFW](url.to/image.jpg) if something is NSFW

2. IDs besides datetime. When you edit a twt then you should preserve the datetime if location-based addressing should have any advantages over content-based addressing. If you change the timestamp the its a new post. Just like any other blog cms.

3. Caching, Yes all good ideas, but that is more a task for the clients not the serving of the twtxt.txt files.

4. Discovery: User-agent for discovery can become better. I’m working on a wrapper script in PHP, so you don’t need to go to Apaches log-files to see who fetches your feed. But for other Gemini and gopher you need to relay on something else. That could be using my webmentions for twtxt suggestion, or simply defining an email metadata field for letting a person know you follow their feed. Interesting read about why WebMetions might be a bad idea. Twtxt being much simple that a full featured IndieWeb sites, then a lot of the concerns does not apply here. But that’s the issue with any open inbox. This is hard to solve without some form of (centralized or community) spam moderation.

5. Support more protocols besides http/s. Yes why not, if we can make clients that merge or diffident between the same feed server by multiples URLs

6. Languages: If the need is big then make a separate feed. I don’t mind seeing stuff in other langues as it is low. You got translating tool if you need to know whats going on. And again when there is a need for easier switching between posting to several feeds, then it’s about building clients with a UI that makes it easy. No something that should takes up space in the format/protocol.

7. Emojis: I’m not sure what this is about. Do you want to use emojis as avatar in CLI clients or it just about rendering emojis?

Righto, @eapl.me@eapl.me, ta for the writeup. Here we go. :-)

Metadata on individual twts are too much for me. I do like the simplicity of the current spec. But I understand where you’re coming from.

Numbering twts in a feed is basically the attempt of generating message IDs. It’s an interesting idea, but I reckon it is not even needed. I’d simply use location based addressing (feed URL + ‘#’ + timestamp) instead of content addressing. If one really wanted to, one could hash the feed URL and timestamp, but the raw form would actually improve disoverability and would not even require a richer client. But the majority of twtxt users in the last poll wanted to stick with content addressing.

yarnd actually sends If-Modified-Since request headers. Not only can I observe heaps of 304 responses for yarnds in my access log, but in Cache.FetchFeeds(…) we can actually see If-Modified-Since being deployed when the feed has been retrieved with a Last-Modified response header before: https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/cache.go#L1278

Turns out etags with If-None-Match are only supported when yarnd serves avatars (https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/handlers.go#L158) and media uploads (https://git.mills.io/yarnsocial/yarn/src/commit/98eee5124ae425deb825fb5f8788a0773ec5bdd0/internal/media_handlers.go#L71). However, it ignores possible etags when fetching feeds.

I don’t understand how the discovery URLs should work to replace the User-Agent header in HTTP(S) requests. Do you mind to elaborate?

Different protocols are basically just a client thing.

I reckon it’s best to just avoid mixing several languages in one feed in the first place. Personally, I find it okay to occasionally write messages in other languages, but if that happens on a more regularly basis, I’d definitely create a different feed for other languages.

Isn’t the emoji thing “just” a client feature? So, feed do not even have to state any emojis. As a user I’d configure my client to use a certain symbol for feed ABC. Currently, I can do a similar thing in tt where I assign colors to feeds. On the other hand, what if a user wants to control what symbol should be displayed, similar to the feed’s nick? Hmm. But still, my terminal font doesn’t even render most of emojis. So, Unicode boxes everywhere. This makes me think it should actually be a only client feature.

@movq@www.uninformativ.de How hard would it be to implement something like (#<2024-10-25T17:15:50Z https://www.uninformativ.de/twtxt.txt>)in jenny as a replacement for (#twthash) and have it not care about if is http(s) or a g-protocol?

Simplified twtxt - I want to suggest some dogmas or commandments for twtxt, from where we can work our way back to how to implement different feature like replies/treads:

1. It’s a text file, so you must be able to write it by hand (ie. no app logic) and read by eye. If you edit a post you change the content not the timestamp. Otherwise it will be considered a new post.

2. The order of lines in a twtxt.txt must not hold any significant. The file is a container and each line an atomic piece of information. You should be able to run sort on a twtxt.txt and it should still work.

3. Transport protocol should not matter, as long as the file served is the same. Http and https are preferred, so it is suggested that feed served via Gopher or Gemini also provide http(s).

4. Do we need more commandments?

@bender@twtxt.net I believe it is Unix-Unix Copy Protocol. Not Unix Copy-Copy Protocol.

@anth@a.9srv.net you wrote:

“Edits and Deletions should go; see also Section 6. This is probably the worst example of this document pushing a text document to do more protocol-like things.”

Edit and deletions are precisely what brought us here. Currently, if one replies to a twtxt, and the original gets later edited, it breaks replies, and potentially drastically changes context.

@prologic@twtxt.net Thanks for writing that up!

I hope it can remain a living document (or sequence of draft revisions) for a good long time while we figure out how this stuff works in practice.

I am not sure how I feel about all this being done at once, vs. letting conventions arise.

For example, even today I could reply to twt abc1234 with “(#abc1234) Edit: …” and I think all you humans would understand it as an edit to (#abc1234). Maybe eventually it would become a common enough convention that clients would start to support it explicitly.

Similarly we could just start using 11-digit hashes. We should iron out whether it’s sha256 or whatever but there’s no need get all the other stuff right at the same time.

I have similar thoughts about how some users could try out location-based replies in a backward-compatible way (append the replyto: stuff after the legacy (#hash) style).

However I recognize that I’m not the one implementing this stuff, and it’s less work to just have everything determined up front.

Misc comments (I haven’t read the whole thing):

• Did you mean to make hashes hexadecimal? You lose 11 bits that way compared to base32. I’d suggest gaining 11 bits with base64 instead.

• “Clients MUST preserve the original hash” — do you mean they MUST preserve the original twt?

• Thanks for phrasing the bit about deletions so neutrally.

• I don’t like the MUST in “Clients MUST follow the chain of reply-to references…”. If someone writes a client as a 40-line shell script that requires the user to piece together the threading themselves, IMO we shouldn’t declare the client non-conforming just because they didn’t get to all the bells and whistles.

• Similarly I don’t like the MUST for user agents. For one thing, you might want to fetch a feed without revealing your identty. Also, it raises the bar for a minimal implementation (I’m again thinking again of the 40-line shell script).

• For “who follows” lists: why must the long, random tokens be only valid for a limited time? Do you have a scenario in mind where they could leak?

• Why can’t feeds be served over HTTP/1.0? Again, thinking about simple software. I recently tried implementing HTTP/1.1 and it wasn’t too bad, but 1.0 would have been slightly simpler.

• Why get into the nitty-gritty about caching headers? This seems like generic advice for HTTP servers and clients.

• I’m a little sad about other protocols being not recommended.

• I don’t know how I feel about including markdown. I don’t mind too much that yarn users emit twts full of markdown, but I’m more of a plain text kind of person. Also it adds to the length. I wonder if putting a separate document would make more sense; that would also help with the length.

I’m still more in favor of (replyto:…). It’s easier to implement and the whole edits-breaking-threads thing resolves itself in a “natural” way without the need to add stuff to the protocol.

I’d love to try this out in practice to see how well it performs. 🤔 It’s all very theoretical at the moment.

More:

Subject: The [tag URI scheme](https://en.wikipedia.org/wiki/Tag_URI_scheme) looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be
        somewhat trivial to parse. But there are still the issue with what the name/id should be... Maybe it doesn't have to bee that stick? Instead of using `tag:` as the prefix/protocol, it would more it clear
        what we are talking about by using `in-reply-to:` (https://indieweb.org/in-reply-to) or `replyto:` similar to `mailto:` 1. `(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)' 2.
        `(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)' 2. `(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)' I know it's longer that 7-11 characters, but it's self-explaining when looking at the
        twtxt.txt in the raw, and the cases above can all be caught with this regex: `\([\w-]*reply[\w-]*\:` Is this something that would work?

Subject: The [tag URI scheme](https://en.wikipedia.org/wiki/Tag_URI_scheme) looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be
        somewhat trivial to parse. But there are still the issue with what the name/id should be... Maybe it doesn't have to bee that stick? Instead of using `tag:` as the prefix/protocol, it would more it clear
        what we are talking about by using `in-reply-to:` (https://indieweb.org/in-reply-to) or `replyto:` similar to `mailto:` 1. `(reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)` 2.
        `(in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)` 3. `(replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)` I know it's longer that 7-11 characters, but it's self-explaining when looking at the
        twtxt.txt in the raw, and the cases above can all be caught with this regex: `\([\w-]*reply[\w-]*\:` Is this something that would work?

Notice the difference? Soren edited, and broke everything.

The tag URI scheme looks interesting. I like that it human read- and writable. And since we already got the timestamp in the twtxt.txt it would be somewhat trivial to parse. But there are still the issue with what the name/id should be… Maybe it doesn’t have to bee that stick?

Instead of using tag: as the prefix/protocol, it would more it clear what we are talking about by using in-reply-to: (https://indieweb.org/in-reply-to) or replyto: similar to mailto:

1. (reply:sorenpeter@darch.dk,2024-09-15T12:06:27Z)
   
2. (in-reply-to:darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
   
3. (replyto:http://darch.dk/twtxt.txt,2024-09-15T12:06:27Z)
   

I know it’s longer that 7-11 characters, but it’s self-explaining when looking at the twtxt.txt in the raw, and the cases above can all be caught with this regex: \([\w-]*reply[\w-]*\:

Is this something that would work?

you can just have a web address.. i added mine.. though i think they have changed up the protocol so my key doesn’t seem to work anymore. https://key.sour.is/id/me@sour.is

I was not suggesting to that everyone need to setup a working webfinger endpoint, but that we take the format of nick+(sub)domain as base for generating the hashed together with the message date and content.

If we omit the protocol prefix from the way we do things now will that not solve most of the problems? In the case of gemini://gemini.ctrl-c.club/~nristen/twtxt.txt they also have a working twtxt.txt at https://ctrl-c.club/~nristen/twtxt.txt … damn I just notice the gemini. subdomain.

Okay what about defining a prefers protocol as part of the hash schema? so 1: https , 2: http 3: gemini 4: gopher ?

@xuu@txt.sour.is Thanks for the link. I found a pdf on one of the authors’ home pages: https://ahmadhassandebugs.github.io/assets/pdf/quic_www24.pdf . I wonder how the protocol was evaluated closer to the time it became a standard, and whether anything has changed. I wonder if network speeds have grown faster than CPU speeds since then. The paper says the performance is around the same below around 600 Mbps.

To be fair, I don’t think QUIC was ever expected to be faster for transferring a single stream of data. I think QUIC is supposed to reduce the impact of a dropped packet by making sure it only affects the stream it’s part of. I imagine QUIC still has that advantage, and this paper is showing the other side of a tradeoff.

So this is a great thread. I have been thinking about this too.. and what if we are coming at it from the wrong direction? Identity being tied to a given URL has always been a pain point. If i get a new URL its almost as if i have a new identity because not only am I serving at a new location but all my previous communications are broken because the hashes are all wrong.

What if instead we used this idea of signatures to thread the URLs together into one identity? We keep the URL to Hash in place. Changing that now is basically a no go. But we can create a signature chain that can link identities together. So if i move to a new URL i update the chain hosted by my primary identity to include the new URL. If i have an archived feed that the old URL is now dead, we can point to where it is now hosted and use the current convention of hashing based on the first url:

The signature chain can also be used to rotate to new keys over time. Just sign in a new key or revoke an old one. The prior signatures remain valid within the scope of time the signatures were made and the keys were active.

The signature file can be hosted anywhere as long as it can be fetched by a reasonable protocol. So say we could use a webfinger that directs to the signature file? you have an identity like frank@beans.co that will discover a feed at some URL and a signature chain at another URL. Maybe even include the most recent signing key?

From there the client can auto discover old feeds to link them together into one complete timeline. And the signatures can validate that its all correct.

I like the idea of maybe putting the chain in the feed preamble and keeping the single self contained file.. but wonder if that would cause lots of clutter? The signature chain would be something like a log with what is changing (new key, revoke, add url) and a signature of the change + the previous signature.

# chain: ADDKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w 
# sig: BEGIN SALTPACK SIGNED MESSAGE. ... 
# chain: ADDURL https://txt.sour.is/user/xuu
# sig: BEGIN SALTPACK SIGNED MESSAGE. ...
# chain: REVKEY kex14zwrx68cfkg28kjdstvcw4pslazwtgyeueqlg6z7y3f85h29crjsgfmu0w
# sig: ...

HTTP/2 differs from 1.x by becoming a binary protocol, it also multiplexes multiple channels over the same connection and has the ability to prefetch related content to the browser to lower the perceived latency.

HTTP/3 moves the binary protocol from HTTP/2 over to QUIC which is based on UDP instead of TCP. This makes it better suited to mobile or unstable networks where handling of transmission errors can be handled at a higher level.

So where do we start wring down the specs/protocol for twtxt2/yarn?

@shreyan@twtxt.net What do you mean when you say federation protocol?

• Either use webfinger for identity like mastodon etc. or use ATproto from Bluesky (or both?)

• We can use webmentions or create our own twt-mentions for notifying someones feed (WIP code at: https://github.com/sorenpeter/timeline/tree/webmention/views)

I’m not sure we need much else. I would not even bother with encryption since other platforms does that better, and for me twtxt/yarn/timeline is for making things public

yarn should define its own federation protocol that extends the basic twtxt in ways that twtxt doesn’t allow. it’s time. and i’ve got ideas!

I would love to see a world where ones twtxt feed is defined by webfinger. So @xuu@txt.sour.is => https://text.sour.is/user/xuu/twtxt.txt

Then my identity can exist independent of the feed location. And I can host multiple protocol types for my feed. Ie. http/gopher/Gemini/irc DCC/etc

I’m also more in favor of #reposts being human readable and writable. A client might implement a bottom that posts something simple like: #repost Look at this cool stuff, because bla bla [alt](url)

This will then make it possible to also “repost” stuff from other platforms/protocols.

The reader part of a client, can then render a preview of the link, which we talked about would be a nice (optional) feature to have in yarnd.

Check out the Nex Protocol. It’s designed to be even simpler than Gemini and Gopher. What do you think? Could be great to host a twtxt feed on.

💡 Quick ‘n Dirty prototype Yarn.social protocol/spec:

If we were to decide to write a new spec/protocol, what would it look like?

Here’s my rough draft (back of paper napkin idea):

• Feeds are JSON file(s) fetchable by standard HTTP clients over TLS
  
• WebFinger is used at the root of a user’s domain (or multi-user) lookup. e.g: prologic@mills.io -> https://yarn.mills.io/~prologic.json
  
• Feeds contain similar metadata that we’re familiar with: Nick, Avatar, Description, etc
  
• Feed items are signed with a ED25519 private key. That is all “posts” are cryptographically signed.
  
• Feed items continue to use content-addressing, but use the full Blake2b Base64 encoded hash.
  
• Edited feed items produce an “Edited” item so that clients can easily follow Edits.
  
• Deleted feed items produced a “Deleted” item so that clients can easily delete cached items.
  

#Yarn.social #Protocol #Ideas

I played around with parsers. This time I experimented with parser combinators for twt message text tokenization. Basically, extract mentions, subjects, URLs, media and regular text. It’s kinda nice, although my solution is not completely elegant, I have to say. Especially my communication protocol between different steps for intermediate results is really ugly. Not sure about performance, I reckon a hand-written state machine parser would be quite a bit faster. I need to write a second parser and then benchmark them.

lexer.go and newparser.go resemble the parser combinators: https://git.isobeef.org/lyse/tt2/-/commit/4d481acad0213771fe5804917576388f51c340c0 It’s far from finished yet.

The first attempt in parser.go doesn’t work as my backtracking is not accounted for, I noticed only later, that I have to do that. With twt message texts there is no real error in parsing. Just regular text as a “fallback”. So it works a bit differently than parsing a real language. No error reporting required, except maybe for debugging. My goal was to port my Python code as closely as possible. But then the runes in the string gave me a bit of a headache, so I thought I just build myself a nice reader abstraction. When I noticed the missing backtracking, I then decided to give parser combinators a try instead of improving on my look ahead reader. It only later occurred to me, that I could have just used a rune slice instead of a string. With that, porting the Python code should have been straightforward.

Yeah, all this doesn’t probably make sense, unless you look at the code. And even then, you have to learn the ropes a bit. Sorry for the noise. :-)

@prologic@twtxt.net On the one hand, twtxt has become more popular thanks to Yarn.social. On the other hand, subject and hashtag extensions took away the simplicity of the protocol. For example, it is impossible to understand which conversation (#base32hash) a tweet refers to or to reply to a tweet without going to a yarn.social pod. Compare with re: in this tweet which can be written without using any client at all

The AT Protocol ?~L~X https://notiz.blog/b/6AM

i find the removal of the “master/slave” thing in computer protocols to be kinkshaming

You need better pen test scripts. :-) Seriously, the protocol is absurdly simple. Turn it on! Don’t trust any of the implementations? Write your own!

It is scary how fast clickhouse can ingest data. Especially if you are using the binary protocol and not the HTTP one.

What if due to climate crisis effects and disasters our digital future will depend on low-energy hardware and protocols like Gemini?

Indeed! I think the first “network protocol client” I ever wrote was something that just did the PING/PONG part and passed everything else raw.

No, totally not useful. 🤣 I mean, the finger protocol is pretty trivial, and it’d be fun to add, but doesn’t replace anything you’re doing.

interesting RFC dated April 1st, 1998: Hyper Text Coffee Pot Control Protocol (HTCPCP/1.0):

https://tools.ietf.org/html/rfc2324

looking at the date this was published, i think the authors originally meant this as an apil’s fool joke/prank.

funny because now we have IOTs and this is somewhat a reality today :P

I’m unclear if I’m going to do the twtxt.net discovery protocol; neither my web server nor Plan 9’s default capture agent strings. :-/

@prologic@twtxt.net the HKP is http keyserver protocol. it’s what happens when you do gpg --send-keys

makes a POST to the keyserver with your pubkey.

Started playing with the gemini protocol on ctrl-c.club

I just installed asuka to try out the gemini protocol (https://gemini.circumlunar.space/)

Ready to explore the gemini protocol as it looks awesome so far

What I need is to serve gopher and http behind a proxy and under the same domain, in a way that a unique container serves each protocol

Enjoying the constraints of the Gopher protocol as a minimalistic zen-mode kind of online publishing revival.

it’s really weird that my most popular medium post thus far this year is a rant about extensions to the gopher protocol

Three protocols and a future of the decentralized internet https://blog.datproject.org/2019/03/22/three-protocols-and-a-future-of-the-decentralized-internet/

1024 possible protocol plateaus

TELNET: The Mother of All (Application) Protocols https://www.ics.uci.edu/~rohit/IEEE-L7-v2.html

Playing with P2P protocols and Firefox http://andregarzia.com/2018/12/playing-with-p2p-protocols-and-firefox.html

Protocol | An ASCII Header Generator for Network Protocols http://www.luismg.com/protocol/

Scuttlebutt Protocol Guide https://ssbc.github.io/scuttlebutt-protocol-guide/

Programming and proving with distributed protocol… https://blog.acolyer.org/2018/01/22/programming-and-proving-with-distributed-protocols/

Re: support for other protocols, it seems like twtxt would be pretty easily adapted to work over the p2p file network DAT, though it’d need client support for DAT or some way to follow people via files and sync in the background, which might be simpler for clients to support but would still require changes to most clients.

Also using curl with txtnish means I can pull from tons of protocols, #gopher #onion…

Additionally, there’s a lot that can be done by a client to reduce the network traffic and UI latency of twtxt without changing the protocol.